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Chapter 1 

The Developmental Project to 
Report TIMSS 2003 Mathematics 
Achievement in Cognitive Domains 

Overview of TIMSS 

TIMSS 2003 is the third and most recently completed round of IEA's 
Trends in International Mathematics and Science Study, a very ambi- 
tious series of international assessments carried out in countries around 
the world to measure trends in mathematics and science learning at the 
fourth and eighth grades. Conducted first in 1995 and then again in 
1999, the regular four-year cycle of TIMSS provides countries with an 
unprecedented opportunity to obtain comparative information about 
their students' achievement in mathematics and science. Forty-nine 
countries participated in TIMSS 2003, with 23 having participated in all 
three assessments and another 14 having participated in two rounds. 
In developing the instruments and procedures for TIMSS 2007, IEA is 
currently working with more than 60 countries. 

The TIMSS 2003 Assessment Frameworks and International Reports 

For TIMSS 2003, the frameworks underlying the mathematics and 
science assessments and questionnaires were updated through a 
major effort. In particular, the mathematics and science frameworks 
were organized along two dimensions - content domains and cogni- 
tive domains. With additional financial support from the US National 
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Science Foundation and the US National Center for Education Statistics, 
IEA's TIMSS & PIRLS International Study Center (ISC) worked with the 
participating countries to describe in detail the mathematics and science 
content to be assessed and to update the learning outcomes related to 
particular cognitive domains. The updated frameworks were published 
in the TIMSS Assessment Frameworks and Specifications 2003, 2nd Edition 
(Mullis, Martin, Smith, Garden, Gregory, Gonzalez, Chrostowski, and 
O'Connor, 2003). 

For mathematics, the five content domains were number, 
algebra (called patterns and relationships at fourth grade), measure- 
ment, geometry, and data. Each content domain described the topic 
areas to be assessed within that domain, and each topic area was elabo- 
rated with objectives specific to the eighth and fourth grades. Four 
cognitive domains were described - Knowing Facts and Procedures, 
Using Concepts, Solving Routine Problems, and Reasoning - together 
with the skills and abilities making up each domain. 

Developing the TIMSS 2003 tests was a cooperative venture 
involving all of the National Research Coordinators (NRCs), includ- 
ing field-testing the items with representative samples of students. 
The NRCs and the Science and Mathematics Item Review Committee 
(SMIRC) had several opportunities to review the items and scoring 
criteria. The resulting TIMSS 2003 mathematics tests contained 194 
items at the eighth grade and 161 items at the fourth grade. 

The international mathematics results from TIMSS 2003 were 
initially reported in the TIMSS 2003 International Mathematics Report: 
Findings from IEA's Trends in International Mathematics and Science Study at 
the Fourth and Eighth Grades (Mullis, Martin, Gonzalezand Chrostowski, 
2004). This report contained overall mathematics achievement results 
for the participating countries as well as achievement in major content 
domains - number, algebra, measurement, geometry, and data. It also 
contained a rich array of information about the school and home con- 
texts for learning mathematics including country-level information 
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collected from the NRCs and considerable data from student, teacher, 
and school questionnaires. 

History of the Developmental Project 

Since the first round of TIMSS in 1995, IEA's TIMSS & PIRLS ISC has 
reported on students' mathematics achievement in content domains 
(e.g., algebra, geometry) and, as noted above, TIMSS 2003 was no 
exception. The TIMSS content domains are fairly consistently found 
in the curricula of the participating countries and the results provide 
an indication of curriculum areas on which students perform rela- 
tively better of worse, both within and across countries. For example, 
TIMSS 1995, 1999, and 2003 have shown that, on average, eighth-grade 
students in the United States perform relatively poorly on geometry 
items and relatively well on data items. For policymakers and educators, 
such information can prove useful in discussions about the curricular 
foci and overall learning goals of students across the country. 

Developing reliable and valid achievement scales for cognitive 
domains can be challenging, since the differences among students across 
and within countries in their mathematics knowledge and problem- 
solving skills make it difficult to know which cognitive abilities students 
are using to solve a given mathematics item. Nevertheless, consider- 
able work has been done in this area by national and international 
assessments, including IEA's Progress in International Reading Literacy 
Study (PIRLS), the OECD's Programme for International Student Assess- 
ment (PISA), and the US National Assessment of Educational Progress 
(NAEP). For example, for the 2004 IEA research conference in Cyprus, 
the TIMSS & PIRLS ISC reported international achievement in the pro- 
cesses of reading comprehension (Mullis, Martin, and Gonzalez, 2004) 
and PIRLS 2006 will institute achievement scales based on processes of 
comprehension. 

Consistent with the growing practice of reporting achievement 
in various cognitive areas, countries participating in TIMSS also have 
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expressed a need for comparative information about how students 
perform in the cognitive domains. To provide enhanced information 
from TIMSS 2003 and facilitate planning for TIMSS 2007, a number 
of participating countries supported a developmental project for IEA's 
TIMSS Er PIRLS International Study Center to examine mathematics 
achievement by cognitive domains. Although focusing on mathematics 
as the first step, if successful the project was intended also to serve as 
a roadmap for achieving similar goals in science. 

Led by the United States, with funding also provided by Chinese 
Taipei, Cyprus, New Zealand, Norway, Ontario, Quebec, Singapore, and 
Sweden, the developmental project involved several major activities. 
Prior to preparing this report of the results of the development study, 
IEA's TIMSS Er PIRLS ISC first convened an international meeting of 
experts in mathematics and mathematics education to confirm the 
mapping of TIMSS 2003 mathematics items to cognitive domains. Then, 
IEA's TIMSS Er PIRLS ISC conducted the various phases of the analytic 
work necessary to create the cognitive domain scale scores. 

Mapping the TIMSS 2003 Mathematics Items to Cognitive Domains 

The developmental project began with a special meeting of mathemat- 
ics experts held in February 2005 in Amsterdam, with the purpose of 
examining the classification of items according to the cognitive domains 
articulated in the TIMSS 2003 mathematics framework. The 10 par- 
ticipants (see Appendix B) expressed great enthusiasm for the meeting 
goal - facilitating TIMSS reporting according to cognitive domains. Nev- 
ertheless, all members expressed reservations about using the cognitive 
domains as they stood. 

In developing the TIMSS 2003 Assessment Framework for 
Mathematics, there were no plans to scale and report results by the 
cognitive domains. In updating the cognitive domains and the learn- 
ing outcomes related to them, the major goal was to encourage item 
writers to be as creative as possible and develop items across a variety 
of cognitive skills and abilities. Although this approach appeared to be 
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viable at the time, and most likely improved the quality of the items 
for TIMSS 2003, it did lead to some overlap across the four cogni- 
tive domains. For example, as demonstrated in assessment items, it 
was sometimes difficult to distinguish between "knowing facts and 
procedures" and "using concepts." This overlap made assigning items 
according to the four original categories very difficult for the members 
of the expert group. As a result, the expert group worked to use the 
existing framework as a basis for developing mutually exclusive cogni- 
tive domains for reporting the TIMSS 2003 results. The process was an 
iterative one involving independent classification of items and discus- 
sion. In classifying items, the expert group followed the guidelines of 
classifying items according to the cognitive process they thought most 
students would use. 

Based on this process and final confirmatory rounds of classify- 
ing the TIMSS 2003 fourth- and eighth-grade items, the experts felt 
comfortable with three cognitive domains: 

• Knowing Facts, Procedures, and Concepts, 

• Applying Knowledge and Understanding, 

• Reasoning. 

The first domain, knowing facts, procedures, and concepts, covers what 
the student needs to know, while the second, applying knowledge and con- 
ceptual understanding, focuses on the ability of the student to apply what 
he or she knows to solve routine problems or answer questions. The third 
domain, reasoning, goes beyond the solution of routine problems to encom- 
pass unfamiliar situations, complex contexts, and multi-step problems. 

Even though all the individuals who participated in the Amster- 
dam Cognitive Domains meeting felt that great progress had been 
made in establishing reliable and valid classifications for analysis and 
reporting, several additional confirmatory steps were taken. First, a 
second expert review was conducted as part of the first TIMSS 2007 
SMIRC meeting held in April 2005. The SMIRC endorsed the work 
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accomplished at the special Mathematics Cognitive Domains meeting 
and worked toward refining the classifications and their descriptions to 
better reflect the essence of the three cognitive domains. This resulted 
in an excellent foundation for scaling the TIMSS 2003 achievement 
data by cognitive domains. 

Also, IEA's TIMSS & PIRLS ISC examined the distribution of 
the items within the three cognitive domains by item type, content 
domain, and difficulty to ensure that there was sufficient coverage of 
each of the newly defined domains. As described in Appendix B (and 
summarized in Exhibit B.l), there was a substantial number of items 
in each domain: 65 in knowing, 93 in applying, and 36 in reasoning 
at eighth grade; and 58 in knowing, 63 in applying, and 38 in reason- 
ing at fourth grade. Within each domain, there was a good spread of 
item type (constructed-response or multiple-choice) at both grades, 
although as might be expected, relatively more of the knowing items 
were multiple choice and relatively more reasoning items constructed 
response. There also was a good spread of items across content domains 
within each of the three cognitive domains, although there was some 
unevenness in some areas. For example, it would have been prefer- 
able to have a higher proportion of number items in the reasoning 
domain at the eighth grade, and a higher proportion of patterns and 
relationship items in the knowing domain and measurement items 
in the reasoning domain at fourth grade. For TIMSS 2007, an effort 
has been made to address these issues in the assessment frameworks. 
Finally, there was a good range of item difficulty within each of the 
cognitive domains, with reasoning items most difficult, on average, as 
would be anticipated. 

The Mathematics Cognitive Domains Framework for the 
TIMSS 2003 Development Project that was used as the basis of this 
report is found in Appendix A. It should be noted that this framework 
was further reviewed by the TIMSS 2007 National Research Coordi- 
nators at their second meeting in Amsterdam, June 2005, resulting in 
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further refinements for TIMSS 2007 as published in the TIMSS 2007 
Assessment Frameworks (Mullis, Martin, Ruddock, O'Sullivan, Arora, and 
Erberber, 2005). 

The Scaling Methodology 

The methodology used to create the mathematics cognitive domain 
scales was identical to that used to report mathematics achievement 
results and achievement in the mathematics content domains in 
the TIMSS 2003 International Reports (Mullis, Martin, Gonzalez & 
Chrostowski, 2004). TIMSS 2003 relied on item response theory scaling 
(IRT) to describe student achievement in mathematics overall, in the 
content domains, and in the cognitive domains. TIMSS created sepa- 
rate scales for mathematics overall, for each content domain, and for 
each cognitive domain at both fourth and eighth grades. The metric 
for the TIMSS overall mathematics scale was established originally in 
TIMSS 1995, with a mean of 500 and standard deviation of 100 across 
the countries participating in that first TIMSS assessment. This was 
done separately for fourth and eighth grades. To provide a mechanism 
for measuring changes in student achievement over time, the data from 
the TIMSS assessments in 1999 (eighth grade only) and 2003 (both 
grades) were finked to this scale. The international average score for 
the eighth-grade countries in 2003 was 467, and for the fourth-grade 
countries, 495. To facilitate comparisons across cognitive domains and 
with overall mathematics, and following the procedure used for the 
mathematics content scales in 2003, the three cognitive domain scales 
were set to have the same mean and standard deviation as the overall 
mathematics scales, i.e., a mean of 467 and standard deviation of 100 
at the eighth grade, and a mean of 495 and standard deviation of 100 at 
the fourth grade. The methodology is summarized in Appendix B and 
is described in detail in the TIMSS 2003 Technical Report (Martin, Mullis 
& Chrostowski, 2004). 
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Summary of Overall Mathematics Achievement Nationally and by 
Gender for the TIMSS 2003 Countries 

To provide a context for considering mathematics achievement at the 
fourth and eighth grades in the cognitive domains, the first page of 
Exhibit 1.1 presents mathematics achievement for all students and 
separately by gender for the 46 countries and four benchmarking 
entities that participated at the eighth grade in TIMSS 2003 and the 
second page presents mathematics achievement in the same way for 
the 25 countries and three benchmarking entities that participated at 
the fourth grade. 1 At each grade, countries are shown in decreasing 
order of average (mean) scale score, together with an indication of 
whether the country average was significantly higher or lower than 
the international average. 2 It should be noted that the results for the 
eighth and fourth grades are not directly comparable. 3 

To recap the overall mathematics achievement results, reported 
in full in the TIMSS 2003 International Mathematics Report , Singapore was 
the highest-performing country at both the fourth and eighth grades. At 
the eighth grade, the Republic of Korea, Hong Kong SAR, and Chinese 
Taipei outperformed all the other countries except Singapore. Japan 
also performed very well, as did Belgium (Flemish), the Netherlands, 
Estonia, and Hungary. At the fourth grade, in addition to Singapore, 
Hong Kong SAR, Japan, and Chinese Taipei also had higher achieve- 
ment than the rest of the countries as did Belgium (Flemish). Belgium 
(Flemish), however, was outperformed by the Asian countries. 

To aid in interpretation, Exhibit 1.1 also includes the years of 
formal schooling and average age of the students in each country. At 
the eighth grade, the aim was that the students assessed would have 
had eight years of formal schooling. Similarly, at the fourth grade, the 
aim was to assess students having had four years of formal school- 
ing. This was the case for most participating countries, however, as 
shown in the TIMSS 2003 International Mathematics Report, the TIMSS 
2003 countries had different policies about the age at which students 
begin formal schooling and about promotion and retention from grade 



1 Details of target population coverage and sampling participation are presented in Appendix C for each country. 

2 The international average of 467 at the eighth grade was obtained by averaging across the mean scores for each of the 46 participating countries. 
The mean scores for the four benchmarking participants were not included in calculating the average. Even though England worked diligently to 
meet the TIMSS sampling requirements and adjustments were made to make the results representative, it did not meet the school participation 
rates as specified in the guidelines and consequently its results are shown below a line. At the fourth grade, the international average of 495 was 
obtained by averaging across the mean scores for the 25 participating countries. 

3 While the scales for the two grades are expressed in the same numerical units, they are not directly comparable in terms of being able to say how 
much achievement or learning at one grade equals how much achievement or learning at the other grade. Comparisons only can be made in 
terms of relative performance. Since the TIMSS scales were developed using IRT technology, like all such scales, the eighth- and fourth-grade scales 
cannot be described in absolute terms. 
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to grade. Thus, even though TIMSS devoted considerable effort to 
maximizing comparability across the grades tested there was some 
variation. Most notably, in the eighth-grade population, students 
in Norway, most of Slovenia, and parts of the Russian Federation 
had fewer years of formal schooling than their counterparts in other 
countries, while those in England, Scotland, New Zealand, and parts 
of Australia had more years of schooling. In the fourth-grade popula- 
tion, some students in Slovenia and parts of the Russian Federation 
had only three years of formal schooling, and students in England and 
Scotland as well as some in Australia and New Zealand had five years. 
Also, equivalence of chronological age does not necessarily mean that 
students have received the same number of year of formal schooling 
or studied the same curriculum. At the eighth grade, students were 
on average between 14 and 15 years old, but the range of policies 
and situations in the participating countries led to considerable varia- 
tion. At the fourth grade, students in most countries were on average 
between 10 and 11 years old. 

As can be seen in the right-hand portion of both pages of 
Exhibit 1.1, at both the eighth and fourth grades, the difference in 
overall mathematics performance by gender was negligible in many 
countries. The situation did vary by country, however. At the eighth 
grade, girls had significantly higher achievement in Singapore, Armenia, 
Serbia, Moldova, Cyprus, Macedonia, Jordan, Bahrain, and the Philip- 
pines. Boys had significantly higher achievement than girls in Belgium 
(Flemish), Hungary, the United States, Italy, Lebanon, Tunisia, Chile, 
Morocco, Ghana, the US state of Indiana, and the Canadian province 
of Quebec. At the fourth grade, girls had significantly higher average 
mathematics achievement in Singapore, Moldova, Armenia, and the 
Philippines. Boys had higher average achievement in the Netherlands, 
the United States, Cyprus, Italy, Scotland, and in the two Canadian 
provinces. 
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TIMSS2003 



Exhibit 1.1: Distribution of Mathematics Achievement Overall and by Gender 



MATHEMATICS 

Grade 



o 

O 



Countries 


Overall 
Average 
Scale Score 


Years of 
Formal 
Schooling* 


Average 

Age 


Singapore 


O 


605 (3.6) 


8 


14.3 


M Korea, Rep. of 


O 


589 (2.2) 


8 


14.6 


t Hong Kong, SAR 


o 


586 (3.3) 


8 


14.4 


Chinese Taipei 


o 


585 (4.6) 


8 


14.2 


Japan 


o 


570 (2.1) 


8 


14.4 


Belgium (Flemish) 


o 


537 (2.8) 


8 


14.1 


t Netherlands 


o 


536 (3.8) 


8 


14.3 


Estonia 


o 


531 (3.0) 


8 


15.2 


Hungary 


o 


529 (3.2) 


8 


14.5 


Malaysia 


o 


508 (4.1) 


8 


14.3 


Latvia 


o 


508 (3.2) 


8 


15.0 


Russian Federation 


o 


508 (3.7) 


7 or 8 


14.2 


Slovak Republic 


o 


508 (3.3) 


8 


14.3 


Australia 


o 


505 (4.6) 


8 or 9 


13.9 


* United States 


o 


504 (3.3) 


8 


14.2 


1 Lithuania 


o 


502 (2.5) 


8 


14.9 


Sweden 


o 


499 (2.6) 


8 


14.9 


t Scotland 


o 


498 (3.7) 


9 


13.7 


2 Israel 


o 


496 (3.4) 


8 


14.0 


New Zealand 


o 


494 (5.3) 


8.5 -9.5 


14.1 


Slovenia 


o 


493 (2.2) 


7 or 8 


13.8 


Italy 


o 


484 (3.2) 


8 


13.9 


Armenia 


o 


478 (3.0) 


8 


14.9 


1 Serbia 


o 


477 (2.6) 


8 


14.9 


Bulgaria 


o 


476 (4.3) 


8 


14.9 


Romania 




475 (4.8) 


8 


15.0 




467 (0.5) 


8 


14.5 


Norway 


® 


461 (2.5) 


7 


13.8 


Moldova, Rep. of 




460 (4.0) 


8 


14.9 


Cyprus 




459 (1.7) 


8 


13.8 


2 Macedonia, Rep. of 


® 


435 (3.5) 


8 


14.6 


Lebanon 


® 


433 (3.1) 


8 


14.6 


Jordan 


® 


424 (4.1) 


8 


13.9 


Iran, Islamic Rep. of 


® 


411 (2.4) 


8 


14.4 


1 Indonesia 


® 


411 (4.8) 


8 


14.5 


Tunisia 


® 


410 (2.2) 


8 


14.8 


Egypt 


® 


406 (3.5) 


8 


14.4 


Bahrain 


® 


401 (1.7) 


8 


14.1 


Palestinian Nat’l Auth. 


® 


390 (3.1) 


8 


14.1 


Chile 


® 


387 (3.3) 


8 


14.2 


1 * Morocco 


® 


387 (2.5) 


8 


15.2 


Philippines 


® 


378 (5.2) 


8 


14.8 


Botswana 


® 


366 (2.6) 


8 


15.1 


Saudi Arabia 


® 


332 (4.6) 


8 


14.1 


Ghana 


® 


276 (4.7) 


8 


15.5 


South Africa 


® 


264 (5.5) 


8 


15.1 


* England 


o 


498 (4.7) 


9 


14.3 


Benchmarking Participants 


Basque Country, Spain 


o 


487 (2.7) 


8 


14.1 


Indiana State, US 


o 


508 (5.2) 


8 


14.5 


Ontario Province, Can. 


o 


521 (3.1) 


8 


13.8 


Quebec Province, Can. 


o 


543 (3.0) 


8 


14.2 



Girls 
Average 
Scale Score 


Boys 
Average 
Scale Score 


Difference 

(Absolute 

Value) 


611 (3.3) 


O 


601 (4.3) 


10 (2.9) 


586 (2.7) 




592 (2.6) 




5 (3.1) 


587 (3.8) 




585 (4.6) 




2 (5.1) 


589 (4.9) 




582 (5.2) 




7 (4.2) 


569 (4.0) 




571 (3.6) 




3 (6.4) 


532 (3.5) 




542 (3.8) 


O 


11 (4.8) 


533 (4.1) 




540 (4.5) 




7 (3.6) 


532 (3.4) 




530 (3.3) 




2 (3.0) 


526 (3.7) 




533 (3.5) 


O 


7 (3.2) 


512 (4.7) 




505 (4.5) 




8 (4.2) 


511 (3.3) 




506 (3.7) 




6 (2.9) 


510 (3.5) 




507 (4.4) 




3 (2.8) 


508 (3.4) 




508 (4.0) 




0 (3.5) 


499 (5.8) 




511 (5.8) 




13 (7.0) 


502 (3.4) 




507 (3.5) 


o 


6(1.9) 


503 (2.9) 




499 (3.0) 




5 (2.9) 


499 (3.0) 




499 (2.7) 




1 (2.2) 


500 (4.3) 




495 (3.8) 




5 (3.5) 


492 (3.3) 




500 (4.5) 




8 (4.0) 


495 (4.8) 




493 (7.0) 




3 (5.7) 


495 (2.6) 




491 (2.6) 




3 (2.8) 


481 (3.0) 




486 (3.9) 


o 


6 (2.8) 


483 (3.3) 


O 


473 (3.4) 




10 (3.0) 


480 (2.9) 


o 


473 (2.9) 




7 (2.8) 


476 (5.5) 




477 (4.3) 




1 (4.7) 


477 (5.1) 




473 (5.0) 




4 (3.3) 


467 (0.6) 


466 (0.6) 


1 (0.6) 


463 (2.7) 




460 (3.0) 




3 (2.8) 


465 (4.1) 


o 


455 (4.8) 




10(3.5) 


467 (1 .9) 


o 


452 (2.3) 




16(2.7) 


439 (4.0) 


o 


431 (3.9) 




9 (3.5) 


429 (3.6) 




439 (3.9) 


o 


10 (4.0) 


438 (4.6) 


o 


411 (5.8) 




27 (6.8) 


417 (4.3) 




408 (4.2) 




9 (7.2) 


411 (4.9) 




410 (5.3) 




1 (3.0) 


399 (2.6) 




423 (2.2) 


o 


24 (1 .9) 


407 (4.4) 




406 (5.0) 




1 (6.4) 


417 (2.4) 


o 


385 (2.4) 




33 (3.3) 


394 (3.9) 




386 (4.7) 




8 (5.9) 


379 (3.5) 




394 (4.3) 


o 


15 (4.5) 


381 (2.8) 




393 (3.0) 


o 


12(3.1) 


383 (5.2) 


o 


370 (5.8) 




13 (3.4) 


368 (2.6) 




365 (2.9) 




3 (1.8) 


326 (7.9) 




336 (5.5) 




10 (9.7) 


266 (5.1) 




283 (4.9) 


o 


17(3.1) 


262 (6.2) 




264 (6.4) 




3 (5.8) 


499 (5.3) 




498 (5.8) 




0 (6.0) 




490 (2.5) 




484 (3.7) 




6 (3.1) 


502 (5.1) 




514 (5.8) 


o 


12 (3.4) 


520 (3.4) 




522 (3.4) 




2 (2.8) 


540 (3.7) 




546 (3.3) 


o 


7 (3.3) 



q Country average significantly higher 
than international average 

(J) Country average significantly lower 
than international average 

* Represents years of schooling counting from the first year of ISCED Level 1 . 
t Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

$ Nearly satisfied guidelines for sample participation rates only after replacement schools were 
included (see Exhibit C.2). 

$ Did not satisfy guidelines for sample participation rates (see Exhibit C.2). 



q Significantly higher than other gender 



1 National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 

2 National Defined Population covers less than 90% of National Desired Population (see Exhibit C.1). 
m Korea tested the same cohort of students as other countries, but later in 2003, at the beginning of 

the next school year. 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 
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TIMSS2003 



Exhibit 1.1: Distribution of Mathematics Achievement Overall and by Gender 



MATHEMATICS 

Grade 



Countries 


Average 
Scale Score 


Years of 
Schooling* 


Average 

Age 


Singapore 


O 


594 (5.6) 


4 


10.3 


t Hong Kong, SAR 


O 


575 (3.2) 


4 


10.2 


Japan 


o 


565 (1.6) 


4 


10.4 


Chinese Taipei 


o 


564 (1.8) 


4 


10.2 


Belgium (Flemish) 


o 


551 (1.8) 


4 


10.0 


t Netherlands 


o 


540 (2.1) 


4 


10.2 


Latvia 


o 


536 (2.8) 


4 


11.1 


1 Lithuania 


o 


534 (2.8) 


4 


10.9 


Russian Federation 


o 


532 (4.7) 


3 or 4 


10.6 


t England 


o 


531 (3.7) 


5 


10.3 


Hungary 


o 


529 (3.1) 


4 


10.5 


t United States 


o 


518 (2.4) 


4 


10.2 


Cyprus 


o 


510 (2.4) 


4 


9.9 


Moldova, Rep. of 




504 (4.9) 


4 


11.0 


Italy 


o 


503 (3.7) 


4 


9.8 


t Australia 




499 (3.9) 


4 or 5 


9.9 




495 (0.8) 


4 


10.3 


New Zealand 




493 (2.2) 


4.5 -5.5 


10.0 


t Scotland 




490 (3.3) 


5 


9.7 


Slovenia 


® 


479 (2.6) 


3 or 4 


9.8 


Armenia 




456 (3.5) 


4 


10.9 


0 Norway 


® 


451 (2.3) 


4 


9.8 


Iran, Islamic Rep. of 


® 


389 (4.2) 


4 


10.4 


Philippines 


® 


358 (7.9) 


4 


10.8 


Morocco 


® 


347 (5.1) 


4 


11.0 


Tunisia 

Benchmarking Participants 


® 


339 (4.7) 


4 


10.4 


Indiana State, US 


o 


533 (2.8) 


4 


9.5 


Ontario Province, Can. 


o 


511 (3.8) 


4 


9.8 


Quebec Province, Can. 


o 


506 (2.4) 


4 


10.1 



Girls 
Average 
Scale Score 


Boys 
Average 
Scale Score 


Difference 

(Absolute 

Value) 


599 (5.5) 


O 


590 (6.2) 




8 (3.9) 


575 (3.4) 




575 (3.4) 




0 (2.3) 


563 (1.8) 




566 (2.1) 




4 (2.3) 


564 (1.7) 




564 (2.1) 




1 (1.7) 


549 (1.8) 




552 (2.5) 




2 (2.5) 


537 (2.7) 




543 (2.2) 


O 


6 (2.4) 


536 (2.9) 




536 (3.5) 




1 (2.9) 


535 (3.5) 




536 (3.2) 




1 (2.8) 


530 (5.4) 




534 (4.7) 




4 (3.5) 


530 (3.9) 




532 (4.5) 




2 (4.0) 


527 (3.8) 




530 (3.3) 




3 (3.4) 


514 (2.4) 




522 (2.7) 


O 


8 (1.6) 


505 (2.7) 




514 (2.9) 


o 


9 (2.8) 


510 (5.2) 


O 


499 (5.1) 




11 (3.5) 


498 (4.1) 




507 (3.7) 


o 


9 (2.6) 


497 (4.5) 




500 (4.3) 




3 (4.0) 


495 (0.8) 


496 (0.8) 


1 (0.7) 


493 (2.7) 




494 (2.4) 




0 (2.9) 


485 (3.2) 




496 (4.4) 


o 


11 (4.1) 


477 (3.0) 




481 (3.5) 




5 (3.8) 


462 (3.7) 


o 


450 (3.8) 




12 (2.9) 


449 (2.7) 




454 (2.7) 




5 (2.8) 


394 (6.5) 




386 (5.5) 




8 (8.8) 


364 (9.2) 


o 


352 (7.0) 




12 (4.6) 


344 (6.1) 




350 (5.1) 




6 (4.7) 


342 (5.0) 




337 (4.9) 




5 (2.8) 


532 (3.1) 




534 (3.4) 




2 (3.3) 


505 (3.6) 




517 (4.7) 


o 


11 (3.7) 


502 (2.7) 




509 (2.8) 


o 


7 (2.7) 



q Country average significantly higher q Significantly higher than other gender 

than international average 

(j) Country average significantly lower 
than international average 



* Represents years of schooling counting from the first year of ISCED Level 1 . 
t Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

1 National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 



0 Norway: 4 years of formal schooling, but First Grade is called "First grade/Preschool." 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 
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Chapter 2 

Mathematics Achievement in the 
Cognitive Domains at the Fourth 
and Eighth Grades 

This chapter of the report presents the TIMSS 2003 mathematics 
achievement results for each of the three cognitive domains. Follow- 
ing the presentation of the results, for each domain in turn - knowing, 
applying, and reasoning - there is an overview of performance 
across domains. 

Knowing Facts, Procedures, and Concepts 

The first page of Exhibit 2.1 presents the distribution of students' 
mathematics achievement in the cognitive domain of knowing facts, 
procedures, and concepts for the 46 countries and four benchmark- 
ing entities that participated in TIMSS 2003 at the eighth grade, and 
the second page presents the distribution of student achievement 
for the 25 countries and three benchmarking entities that partici- 
pated at the fourth grade. Countries are shown in decreasing order 
of average (mean) scale score, together with an indication of whether 
the country average is significantly higher or lower than the interna- 
tional average. To provide a basis of comparison for the performance 
of each country in each cognitive domain, the international average 
across countries for each domain was scaled to be 467, the same as 
the international average for mathematics overall. As explained in 
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Chapter 1 (footnote 1), the benchmarking entities were not included 
in computing the international average. Also, as previously discussed 
in conjunction with Exhibit 1.1, the years of formal schooling and 
average age of the students in each country are shown to aid in inter- 
pretation of the achievement results. This information also is repeated 
in Exhibit 2.1 as well as in 2.3 and 2.5 for ease of reference. 

Finally, as a reminder that not all countries are equally well 
equipped to meet the challenge of educating their young people, 
Exhibit 2.1 as well as Exhibits 2.3 and 2.5 include the value for each 
country on the Human Development Index provided by the United 
Nations Development Programme (see Human Development Report 2003) . 
The index has a minimum value of 0 and a maximum of 1.0. Coun- 
tries with high values on the index enjoy long life expectancy, high 
levels of school enrollment and adult literacy, and a good standard of 
living as measured by per capita GDP. For example, at the eighth grade, 
TIMSS countries with index values greater than 0.9 included Aus- 
tralia, Belgium (Flemish), England, Israel, Italy, Japan, New Zealand, 
Norway, The Netherlands, Scotland, Sweden, and the United States. 
For all three cognitive domains, all of these countries (except Norway 
in the knowing and applying domains) had average achievement above 
the international average. However, not all countries performing above 
the overall international average in the three cognitive domains had 
an index value as high as 0.9. Within each of the cognitive domains, 
the relationship between a country's index value and average student 
achievement was fairly similar. 

As shown in Exhibit 2.1, in the knowing domain for the eighth 
grade, similar to overall mathematics performance, there was a wide 
range in performance between the highest- and lowest-performing 
countries, from 592 in the Republic of Korea to 232 in Ghana. Twenty- 
seven countries and the four benchmarking entities performed above 
the international average and 17 countries scored below the interna- 
tional average. Moldova and Cyprus performed about the same as the 
international average. 
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At the fourth grade, the difference was also large between the 
highest-performing country Singapore (626) and the lowest-performing 
country Tunisia (338). Thirteen countries and the three benchmarking 
entities performed above the international average and eight countries 
performed below the international average. The four countries per- 
forming about at the international average were Australia, Moldova, 
Cyprus, and New Zealand. 

For both the eighth and fourth grades, Exhibit 2.1 illustrates 
the broad range of achievement both within and across the countries 
assessed. It provides a graphical representation of student performance 
within each country. The bar graph for each country shows the 5th, 
25th, 75th, and 95th percentiles 1 as well as the 95% confidence for the 
mean. Each percentile point indicates the percentage of students below 
that point on the scale. For most TIMSS 2003 participants at the eighth 
grade, there was an enormous range within each country between the 
highest and lowest scores, often as much as 400 scale-score points. This 
range was as large or larger than the difference in mean achievement 
between the highest and lowest performing country. For the eighth 
grade knowing scale, the range for most students in the higher- achiev- 
ing countries was from 400 to 700. In comparison, it tended to be 
between 300 and 600 for medium-performing countries and from 200 
to 500 (or even lower) in the lower-performing countries. 

Exhibit 2.2 shows how a country's average mathematics achieve- 
ment in the knowing domain compares to achievement in the other 
participating countries. The results for the eighth grade are shown on 
the first two pages and for the fourth grade on the third page. The 
figure for each grade shows whether or not the differences in average 
achievement between pairs of countries are statistically significant. To 
read the table, select a country of interest from the first column and 
read across the row corresponding to that country. A circle with a tri- 
angle pointing up indicates significantly higher performance than the 
comparison country listed across the top; absence of a symbol indicates 



1 Tables of the percentiles values and standard deviations for all countries are presented in Appendix D. 
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Exhibit 2.1: Distribution of Mathematics Achievement for Knowing Cognitive Domain 



MATHEMATICS 

Grade 




Countries 


Years of 


Average 


Schooling* 


Age 


M Korea, Rep. of 


8 


14.6 


Singapore 


8 


14.3 


t Hong Kong, SAR 


8 


14.4 


Chinese Taipei 


8 


14.2 


Japan 


8 


14.4 


Estonia 


8 


15.2 


Belgium (Flemish) 


8 


14.1 


Hungary 


8 


14.5 


t Netherlands 


8 


14.3 


Russian Federation 


7 or 8 


14.2 


Latvia 


8 


15.0 


Slovak Republic 


8 


14.3 


1 Lithuania 


8 


14.9 


* United States 


8 


14.2 


Malaysia 


8 


14.3 


2 Israel 


8 


14.0 


Slovenia 


7 or 8 


13.8 


Australia 


8 or 9 


13.9 


1 Serbia 


8 


14.9 


Sweden 


8 


14.9 


Bulgaria 


8 


14.9 


Romania 


8 


15.0 


New Zealand 


8.5 -9.5 


14.1 


Italy 


8 


13.9 


t Scotland 


9 


13.7 


Armenia 


8 


14.9 




8 


14.5 


Moldova, Rep. of 


8 


14.9 


Cyprus 


8 


13.8 


Norway 


7 


13.8 


Lebanon 


8 


14.6 


2 Macedonia, Rep. of 


8 


14.6 


Jordan 


8 


13.9 


1 Indonesia 


8 


14.5 


Egypt 


8 


14.4 


Iran, Islamic Rep. of 


8 


14.4 


Bahrain 


8 


14.1 


Tunisia 


8 


14.8 


Palestinian Nat'l Auth. 


8 


14.1 


Philippines 


8 


14.8 


1 * Morocco 


8 


15.2 


Chile 


8 


14.2 


Botswana 


8 


15.1 


Saudi Arabia 


8 


14.1 


South Africa 


8 


15.1 


Ghana 


8 


15.5 


* England 


9 


14.3 


Benchmarking Participants 


Basque Country, Spain 


8 


14.1 


Indiana State, US 


8 


14.5 


Ontario Province, Can. 


8 


13.8 


Quebec Province, Can. 


8 


14.2 




Average 
Scale Score 


Human 

Development 

Index** 


592 (2.1) 


O 


0.879 


591 (3.1) 


O 


0.884 


589 (3.3) 


o 


0.889 


585 (4.5) 


o 


- 


564 (1.9) 


o 


0.932 


538 (2.7) 


o 


0.833 


537 (2.5) 


o 


0.937 


536 (3.1) 


o 


0.837 


520 (3.1) 


o 


0.938 


519 (3.4) 


o 


0.779 


518 (2.8) 


o 


0.811 


517 (3.3) 


o 


0.836 


511 (2.7) 


o 


0.824 


510 (2.8) 


o 


0.937 


506 (3.9) 


o 


0.790 


501 (3.1) 


o 


0.905 


499 (2.2) 


o 


0.881 


497 (4.0) 


o 


0.939 


495 (2.7) 


o 


- 


486 (2.1) 


o 


0.941 


486 (4.1) 


o 


0.795 


485 (4.9) 


o 


0.773 


485 (4.8) 


o 


0.917 


484 (3.2) 


o 


0.916 


481 (3.2) 


o 


0.930 


480 (2.9) 


o 


0.729 


467 (0.5) 


- 


466 (4.1) 




0.700 


466 (2.0) 




0.891 


450 (2.1) 




0.944 


447 (3.2) 




0.752 


447 (3.8) 


® 


0.784 


428 (4.7) 


® 


0.743 


422 (4.3) 


® 


0.682 


411 (3.4) 


® 


0.648 


405 (2.6) 


® 


0.719 


401 (2.3) 


® 


0.839 


399 (3.0) 


® 


0.740 


391 (3.7) 


® 


0.731 


388 (5.2) 


® 


0.751 


386 (2.8) 


® 


0.606 


386 (3.2) 


® 


0.831 


372 (2.8) 


® 


0.614 


315 (4.6) 


® 


0.769 


261 (5.4) 


® 


0.684 


232 (5.9) 


® 


0.567 


489 (4.0) 


o 


0.930 




495 (2.2) 


o 


- 


515 (4.6) 


o 


- 


513 (2.6) 


o 


- 


537 (2.7) 


o 


- 



0 100 200 300 400 500 600 700 800 



Percentiles of Performance 

5th 25th ■ 75th | 95th 

95% Confidence Interval for Average (±2SE) 



o Country average significantly higher 
than international average 

® Country average significantly lower 
than international average 



Represents years of schooling counting from the first year of ISCED Level 1 . 

Taken from United Nations Development Programme's Human Development Report 2003, 
p. 237-240. 

Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

Nearly satisfied guidelines for sample participation rates only after replacement schools were 
included (see Exhibit C.2). 

Did not satisfy guidelines for sample participation rates (see Exhibit C.2). 



1 National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 

2 National Defined Population covers less than 90% of National Desired Population (see Exhibit C.1). 
m Korea tested the same cohort of students as other countries, but later in 2003, at the beginning of 

the next school year. 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 

A dash (-) indicates comparable data are not available. 
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Exhibit 2.1: Distribution of Mathematics Achievement for Knowing Cognitive Domain 



MATHEMATICS 

Grade 



Countries 


Years of 
Schooling* 


Average 

Age 


Mathematics Achievement Distribution 




Average 
Scale Score 


Human 

Development 

Index** 


Singapore 


4 


10.3 






626 (6.5) O 


0.884 



t Hong Kong, SAR 
Chinese Taipei 
Japan 

Belgium (Flemish) 
t England 
t Netherlands 
t United States 
1 Lithuania 
Hungary 
Latvia 
Italy 

Russian Federation 
t Australia 
Moldova, Rep. of 
Cyprus 




574 (3.3) 


O 


0.889 


565 (2.2) 


O 


- 


564 (2.1) 


o 


0.932 


558 (2.1) 


o 


0.937 


534 (4.5) 


o 


0.930 


530 (2.2) 


o 


0.938 


528 (2.5) 


o 


0.937 


519 (2.7) 


o 


0.824 


517 (3.3) 


o 


0.837 


517 (2.9) 


o 


0.811 


514 (3.9) 


o 


0.916 


513 (5.3) 


o 


0.779 


501 (3.8) 




0.939 


500 (5.2) 




0.700 


500 (2.8) 




0.891 









New Zealand 4.5 - 5.5 10.0 493 (2.2) 0.917 



t Scotland 5 9.7 




484 (3.0) ® 0.930 


Slovenia 3 or 4 9.8 




470 (2.6) ® 0.881 


0 Norway 4 9.8 




448 (2.1) ® 0.944 


Armenia 4 10.9 




447 (3.7) ® 0.729 


Iran, Islamic Rep. of 4 10.4 


- 


404 (4.0) ® 0.719 


Philippines 4 10.8 


— - 3! — 


385 (6.9) ® 0.751 


Morocco 4 11.0 


_ . _ 


360 (4.4) ® 0.606 


Tunisia 4 10.4 


— s 


338 (4.2) ® 0.740 


Benchmarking Participants 






Indiana State, US 4 10.5 




544 (3.7) O 


Ontario Province, Can. 4 9.8 


- 


514 (4.4) O 


Quebec Province, Can. 4 10.1 




504 (2.8) O 



0 100 200 300 400 500 600 700 800 



5th 



Percentiles of Performance — 
25th ■ 75th 



95% Confidence Interval for Average (±2SE) 



O Country average significantly higher 
than international average 

(j) Country average significantly lower 
than international average 



Represents years of schooling counting from the first year of ISCED Level 1 . 

Taken from United Nations Development Programme's Human Development Report 2003, 
p. 237-240. 

Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 



0 Norway: 4 years of formal schooling, but First Grade is called "First grade/Preschool." 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 

A dash (-) indicates comparable data are not available. 
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Exhibit 2.2: Multiple Comparisons of Average Mathematics Achievement for Knowing 

Cognitive Domain 



MATHEMATICS 

Grade 




Instructions: Read across the row for a country to compare performance with the countries listed along the top of the chart. The symbols indicate whether the average 
achievement of the country in the row is significantly lower than that of the comparison country, significantly higher than that of the comparison country, or if there is no 
statistically significant difference between the average achievement of the two countries. 



Countries 


Korea, Rep. of 


Singapore 


Hong Kong, SAR 


Chinese Taipei 


Japan 


Estonia 


Belgium (Flemish) 


Hungary 


Netherlands 


Russian Federation 


Latvia 


Slovak Republic 


Lithuania 


United States 


Malaysia 


Israel 


Slovenia 


Australia 


Serbia 


England 


Sweden 


Bulgaria 


Romania 


New Zealand 


Italy 


Scotland 


Armenia 


Moldova, Rep. of 


Cyprus 


Norway 


Korea, Rep. of 


■ 


■ 






O 


O 




o 




O 


O 






o 


O 


O 


o 


o 




O 


o 


o 


o 


o 


O 


O 






0 


o 


Singapore 






■ 




° 


O 


O 


O 


O 


° 


o 


O 


O 


O 


O 


o 


O 


O 


O 


O 


o 


O 


O 


O 


° 


o 


O 


O 


o 


° 


Hong Kong, SAR 




■ 






o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


Chinese Taipei 




■ 






° 


o 


o 


o 


o 


° 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


0 


o 


o 


o 


o 


° 


Japan 


® 


® 


® 


® 




o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


Estonia 




® 


® 


® 










o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


o 


Belgium (Flemish) 


® 


® 


® 


® 


® 








® 


o 


o 




0 




o 


o 








o 


o 








o 


o 






0 


o 


Hungary 


® 


® 


® 


® 


® 








o 


° 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


0 


o 


o 


o 


o 


° 


Netherlands 


® 


® 


® 


® 


® 


® 


® 


® 










o 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


0 


o 


Russian Federation 


® 


® 


® 


® 


® 


® 


® 


® 










o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


0 


o 


o 


o 


o 




Latvia 


® 


® 


® 


® 


® 


® 


® 


® 










° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


0 


o 


Slovak Republic 


® 


® 


® 


® 


® 


® 


® 


® 






o 


o 


o 


o 


o 


o 


o 


o 


o 


0 


o 


o 


o 


o 


° 


Lithuania 


® 


® 


® 


® 


® 


® 


® 


® 


® 






o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


United States 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


o 


o 


o 


o 


o 


o 


o 


o 


o 


0 


o 


o 


o 


o 


° 


Malaysia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 














° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


0 


o 


Israel 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 










o 


o 


o 


o 


o 


0 


o 


o 


o 


o 


° 


Slovenia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 






o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


Australia 


® 


® 


® 


® 




® 


® 


® 


® 


® 


® 


® 


® 


® 






o 








° 


o 


o 


o 


o 


° 


Serbia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 




o 


° 






o 


o 


° 


° 


° 


o 


England 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


■ 


■ 










o 


o 


° 


Sweden 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 










° 


° 


o 


Bulgaria 


® 


® 


® 


® 




® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 




® 










o 


o 


° 


Romania 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 














° 


° 


o 


New Zealand 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 








■ 


■ 


■ 


■ 


■ 






o 


o 


° 


Italy 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


















° 


° 


o 


Scotland 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 








■ 


0 


o 


° 


Armenia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


















° 


0 


o 


Moldova, Rep. of 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 






O 


Cyprus 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


• 




® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


®\ 


® 






o 


Norway 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


■ 


Lebanon 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


0 


0 


0 




Macedonia, Rep. of 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


■ 


® 


® 


® 


® 




Jordan 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


'l®\ 


0 


0 


0 


® 


Indonesia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


Egypt 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


0 


0 


0 


® 


Iran, Islamic Rep. of 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


Bahrain 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


'l®\ 


0 


0 


0 


® 


Tunisia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


Palestinian Nat'l Auth. 


® 


0 


® 


® 


® 


® 


0 


0 


0 


® 


® 


2 


® 


0 


® 


® 


0 


0 


0 


® 


® 


0 


0 


0 


® 


® 


0 


0 


0 


® 


Philippines 


® 


® 


® 


® 


® 


® 


® 


® 


® 




® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


Morocco 


® 


0 


® 


® 


® 


1 ® 


0 


0 


0 


® 


® 


® 


® 


0 


® 


^§TI 


0 


0 


|®i 


r® 


® 


0 


0 


0 


® 


'l®\ 


0 


0 


0 


® 


Chile 


® 


® 


® 


® 


® 


® 


® 


® 


® 




® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


Botswana 




0 


® 


® 


® 


®\ 


0 


0 


0 


® 




® 


® 


0 


® 


®\ 


0 


0 


0 


\-® 


® 


0 


0 


0 


® 




0 


0 


0 


® 


Saudi Arabia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


South Africa 


^® 


0 


® 


® 


® 


1 ® 


0 


0 


0 


® 


® 


® 


® 


0 


® 


® 


0 


0 


0 


-® 


® 


0 


0 


0 


® 


1 ® 


0 


0 


0 


® 


Ghana 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


Benchmarking Participants 










Basque Country, Spain 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 




o 


o 






° 


o 


o 


o 


o 


° 


Indiana State, US 


® 


0 


0 


0 


® 




0 


0 






o 


0 


0 


0 


o 


o 


0 




0 


o 


o 


0 


o 


0 


o 


Ontario Province, Can. 


® 


® 


® 


® 


® 


® 


® 


® 


■■■■■■ 




o 


O 


o 


o 


o 


o 


o 


O 


o 


O 


o 


o 


o 


o 


® 


Quebec Province, Can. 


® 


[®1 


f®l 


[®1 


® 








o 


o 


° 


° 


o 




o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 



Note: 5% of these comparisons would be statistically significant by chance alone. 
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Exhibit 2.2: Multiple Comparisons of Average Mathematics Achievement for Knowing 

Cognitive Domain 



MATHEMATICS 

Grade 




Instructions: Read across the row for a country to compare performance with the countries listed along the top of the chart. The 
symbols indicate whether the average achievement of the country in the row is significantly lower than that of the comparison country, 
significantly higher than that of the comparison country, or if there is no statistically significant difference between the average 
achievement of the two countries. 



Lebanon 


Macedonia, Rep. of 


Jordan 


Indonesia 


Egypt 


Iran, Islamic Rep. of 


Bahrain 


Tunisia 


Palestinian Nat'l Auth. 


Philippines 


Morocco 


Chile 


Botswana 


Saudi Arabia 


South Africa 


Ghana 




Basque Country, Spain 


Indiana State, US 


Ontario Province, Can. 


Quebec Province, Can. 


Countries 


O 


0 


0 


0 


o 


O 




o 


o 


O 


o 


o 






o 


o 




O 






o 


Korea, Rep. of 




O 


O 


O 


O 


o 


o 


o 


o 


O 


o 


° 


o 


O 


O 


O 


° 


■ 


o 


O 


O 


O 


Singapore 


■ 


o 


0 


0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 




o 


° 


° 


° 


Hong Kong, SAR 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 


o 


o 


o 


o 


Chinese Taipei 


■ 


o 


0 


0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 




o 


° 


° 


° 


Japan 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 


o 


o 


o 




Estonia 


■ 


o 


0 


0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 




o 


° 


° 




Belgium (Flemish) 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 


o 


o 


o 




Hungary 


■ 


o 


0 


0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 




o 






® 


Netherlands 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


O 


o 


o 


o 


o 


° 


■ 


o 






® 


Russian Federation 


■ 


o 








o 


o 


® 


® 


® 


o 


o 


® 


® 


® 


® 


o 




o 






® 


Latvia 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


O 


o 


o 


o 


o 


° 


■ 


o 






® 


Slovak Republic 


■ 


o 




o 


o 


o 


o 




® 


® 


o 


o 


® 


® 


® 


® 


o 




o 






• 


Lithuania 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


O 


o 


o 


o 


o 


° 


■ 


o 






® 


United States 


■ 


o 








o 


o 




® 


® 


o 


o 


® 


® 


® 


® 


o 




o 






® 


Malaysia 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


0 


o 


o 


o 


o 


0 


■ 




® 


® 


® 


Israel 


■ 


o 


0 


0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 






® 


® 


® 


Slovenia 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 




® 


® 


® 


Australia 


■ 


o 


0 


0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 






® 


® 


® 


Serbia 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 




® 


® 


® 


England 


■ 


o 


0 


0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 




® 


® 


® 


® 


Sweden 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 


® 


® 


® 


® 


Bulgaria 


■ 


o 


0 


0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 






® 


® 


® 


Romania 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 




® 


® 


® 


New Zealand 


■ 


o 


0 


0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 




® 


® 


® 


® 


Italy 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 


® 


® 


® 


® 


Scotland 


■ 


o 


0 


0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 




® 


® 


® 


® 


Armenia 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 


® 


® 


® 


® 


Moldova, Rep. of 


■ 


o 


o 


0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 




® 


® 


® 


® 


Cyprus 








o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 


® 


® 


® 


® 


Norway 


■ 






0 


0 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 




® 


® 


® 


® 


Lebanon 








0 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 


® 


® 


® 


® 


Macedonia, Rep. of 


■ 


® 


® 






o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


° 


o 




® 


® 


® 


® 


Jordan 




® 


® 






o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


° 


■ 


® 


® 


® 


® 


Indonesia 


■ 


®\ 


0 




0 






° 


° 


° 


o 


o 


° 


° 


° 


° 


o 




® 


® 


® 


® 


Egypt 




® 


® 


® 


® 










o 


o 


° 


o 


o 


o 


o 


° 


■ 


® 


® 


® 


® 


Iran, Islamic Rep. of 


■ 


®\ 


0 


0 


0 










° 


o 


o 


° 


° 


° 


° 


o 




® 


® 


® 


® 


Bahrain 




® 


® 


® 


® 


® 












° 


o 


o 


o 


o 


° 


■ 


® 


® 


® 


® 


Tunisia 


■ 


®] 


0 


0 


0 


\® 


®j 














° 


0 


° 


o 




® 


® 


1® 


E 


Palestinian Nat'l Auth. 




® 


® 


® 


® 


® 


® 


® 










■ 


o 


o 


o 




■ 


® 


® 


® 


® 


Philippines 


■ 


®\ 


0 


0 


0 


1 ® 


®j 


0 


® 










o 


° 


° 


o 




® 


® 


® 


® 


Morocco 




® 


® 


® 


® 


® 


® 


® 


® 










o 


o 


o 


° 


■ 


® 


® 


® 


® 


Chile 


■ 


®\ 


0 


0 


0 


r® 


®l 


0 


0 


0 


1® 


® 


* 




° 


° 


o 




® 


® 


® 


® 


Botswana 




® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 




o 


° 


■ 


® 


® 


® 


® 


Saudi Arabia 


■ 


~®~ 


0 


0 


0 


® 


®j 


0 


0 


0 


i® 


® 


0 


® 


’ ® 




o 




® J 


® 


® 


® 


South Africa 




® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


■ 


■ 


® 


® 


® 


® 


Ghana 


■ 



Benchmarking Participants 



O 


o 


o 




o 


o 
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Indiana State, US 




o 
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o 


o 


o 




o 


o 


o 
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° 
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o 
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® 


Ontario Province, Can. 




o 
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o 


o 


o 
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o 


o 


o 


o 


o 


o 




o 




o 




o 

o 


° 




Quebec Province, Can. 





O Average achievement 
significantly higher than 
comparison country 



Average achievement 
significantly lower than 
comparison country 



Note: 5% of these comparisons would be statistically significant by chance alone. 
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Exhibit 2.2: 



Multiple Comparisons of Average Mathematics Achievement for Knowing 
Cognitive Domain 



MATHEMATICS 

Grade 




Instructions: Read across the row for a country to compare performance with the countries listed along the top of the chart. The symbols indicate 
whether the average achievement of the country in the row is significantly lower than that of the comparison country, significantly higher than 
that of the comparison country, or if there is no statistically significant difference between the average achievement of the two countries. 



Countries 


Singapore 


Hong Kong, SAR 


Chinese Taipei 


Japan 


Belgium (Flemish) 
England 


Netherlands 


United States 


Lithuania 


Hungary 

Latvia 


Italy 


Russian Federation 


Australia 


Moldova, Rep. of 
Cyprus 


New Zealand 


Scotland 


Slovenia 


Norway 


Armenia 


Iran, Islamic Rep. of 


Philippines 


Morocco 




Indiana State, US 


Ontario Province, Can. 


Quebec Province, Can. 


Singapore 
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O O 


0 


0 
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O O 
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O O 
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0 
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O 
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0 


0 
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O 


0 


0 
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0 
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o 


o 
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o 
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o 
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o 


o 
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o 
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o 
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® 
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Lithuania 
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® 
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® 


® ® 


® 
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0 
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® 


® ® 


® 
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■ 
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® 


® 


® 


® 


® ® 


® 


® 
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o o 


0 


0 
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o 


0 


0 


0 
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® 
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Italy 


® 


® 


® 


® 


® ® 


® 


® 
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o 
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Russian Federation 


® 
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® ® 


® 
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Australia 


® 
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® ® 


® 
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® ® 
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®\ 
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Iran, Islamic Rep. of 
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® ® 
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® ® 
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® 


® ® 


® 
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® 
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®| 
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® 
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1 ®j 








n 


® 


o 
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O Average achievement 
significantly higher than 
comparison country 



(§> Average achievement 
significantly lower than 
comparison country 



Note: 5% of these comparisons would be statistically significant by chance alone. 
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no significant difference in performances; and a circle with a triangle 
pointing down indicates significantly lower performance. 

At the eighth grade, the Republic of Korea, Singapore, Hong 
Kong SAR, and Chinese Taipei had significantly higher achievement in 
the knowing domain than the other participating countries. With the 
exception of those four top-performing countries, Japan had signifi- 
cantly higher achievement than all the rest of the participating coun- 
tries. Estonia, Belgium (Flemish), and Hungary also performed very 
well as did the Canadian province of Quebec, being outperformed by 
only the five top-scoring Asian countries. 

At the fourth grade, Singapore had the highest average achieve- 
ment in the knowing domain followed by Hong Kong SAR and then 
Chinese Taipei and Japan. Belgium (Flemish) outperformed all the 
participating countries except the four top-scoring Asian countries. 
England, the Netherlands, the United States, and the US state of 
Indiana also had higher average achievement than many of the other 
participating countries. 

Applying Knowledge and Conceptual Understanding 

Exhibit 2.3 presents the distribution of student mathematics achieve- 
ment in the cognitive domain of applying at the eighth (first page) and 
fourth (second page) grades. 

At the eighth grade, led by Singapore, 24 countries and the four 
benchmarking participants had achievement in the applying domain 
significantly higher than the international average. Romania, Bulgaria, 
Norway, and Serbia performed no differently than the international 
average and 18 countries performed significantly below this average. At 
the fourth grade, also led by Singapore, 14 countries and the US state 
of Indiana had achievement significantly higher than the international 
average, two countries (Italy and Australia) and the two Canadian 
provinces had achievement similar to the international average, and 9 
countries had achievement below it. 
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Exhibit 2.3: Distribution of Mathematics Achievement for Applying Cognitive Domain 



MATHEMATICS 

Grade 




Countries 


Years of 
Schooling* 


Average 

Age 


Mathematics Achievement Distribution 1 


Average 
Scale Score 


Human pH 

Development Bsj 
Index** H 


Singapore 


8 


14.3 


J wmmm 


611 (3.6) 


O 


0.884 ■§ 


t Hong Kong, SAR 


8 


14.4 




584 (3.2) 


O 


0.889 


M Korea, Rep. of 


8 


14.6 




584 (2.2) 


o 


0.879 g 


Chinese Taipei 


8 


14.2 


^ 


582 (4.6) 


o 




Japan 


8 


14.4 




564 (2.2) 


o 


0.932 


t Netherlands 


8 


14.3 




543 (3.7) 


o 


0.938 | 


Belgium (Flemish) 


8 


14.1 




536 (2.7) 


o 


0.937 | 


Estonia 


8 


15.2 




528 (2.9) 


o 


0.833 X 


Hungary 


8 


14.5 




523 (3.4) 


o 


0.837 | 


Malaysia 


8 


14.3 


;E — 


512 (4.4) 


o 


0.790 | 


Australia 


8 or 9 


13.9 


- 


508 (4.8) 


o 


0.939 


t Scotland 


9 


13.7 


— ;c — ■ 


505 (3.9) 


o 


0.930 J 


Sweden 


8 


14.9 


■ 


505 (2.8) 


o 


0.941 


Latvia 


8 


15.0 


— — 


504 (3.4) 


o 


o.8n | 


Russian Federation 


7 or 8 


14.2 


— ■: — 


503 (3.7) 


o 


0.779 £ 


Slovak Republic 


8 


14.3 


— — 


502 (3.7) 


o 


0.836 ty 


* United States 


8 


14.2 


— * — 


502 (3.4) 


o 


0.937 u 


1 Lithuania 


8 


14.9 


—— — 


499 (2.8) 


o 


0.824 


New Zealand 


8.5 -9.5 


14.1 


— - — 


497 (5.3) 


o 


0.917 


2 Israel 


8 


14.0 




495 (3.6) 


o 


0.905 


Slovenia 


7 or 8 


13.8 


— 3 


491 (2.3) 


o 


0.881 


Italy 


8 


13.9 




484 (3.2) 


o 


0.916 


Armenia 


8 


14.9 




478 (3.0) 


o 


0.729 


Romania 


8 


15.0 




475 (5.0) 




0.773 


Bulgaria 


8 


14.9 




471 (4.7) 




0.795 


Norway 


7 


13.8 




468 (2.7) 




0.944 


1 Serbia 


8 


14.9 




467 (2.9) 




- 


International Avg. 


8 


14.5 




467 (0.5) 


o 


Moldova, Rep. of 


8 


14.9 




457 (3.9) 




0.700 


Cyprus 


8 


13.8 




457 (1.6) 




0.891 


2 Macedonia, Rep. of 


8 


14.6 




428 (3.8) 


® 


0.784 


Lebanon 


8 


14.6 




426 (3.3) 


® 


0.752 


Jordan 


8 


13.9 




422 (4.2) 


® 


0.743 


Tunisia 


8 


14.8 


IKH 


419 (2.3) 


® 


0.740 


Iran, Islamic Rep. of 


8 


14.4 


■: — 


416 (2.5) 


® 


0.719 


1 Indonesia 


8 


14.5 




408 (4.9) 


® 


0.682 


Egypt 


8 


14.4 




404 (3.4) 


® 


0.648 


Bahrain 


8 


14.1 


■mm ■' mmmm 


398 (1.6) 


® 


0.839 


Chile 


8 


14.2 


. hi 


391 (3.3) 


® 


0.831 


Palestinian Nat'l Auth. 


8 


14.1 




388 (3.2) 


® 


0.731 


1 * Morocco 


8 


15.2 




384 (2.9) 


® 


0.606 


Philippines 


8 


14.8 




378 (4.8) 


® 


0.751 


Botswana 


8 


15.1 




369 (2.7) 


® 


0.614 


Saudi Arabia 


8 


14.1 




338 (3.6) 


® 


0.769 


Ghana 


8 


15.5 


— 


293 (4.0) 


® 


0.567 


South Africa 


8 


15.1 




269 (5.3) 


® 


0.684 


* England 


9 


14.3 




503 (4.8) 


o 


0.930 


Benchmarking Participants 




Basque Country, Spain 


8 


14.1 


— . - ■ 


481 (2.3) 


o 


- 


Indiana State, US 


8 


14.5 




507 (5.9) 


o 


- 


Ontario Province, Can. 


8 


13.8 




522 (3.0) 


o 


- 


Quebec Province, Can. 


8 


14.2 




545 (3.0) 


o 


- 



0 100 200 300 400 500 600 700 800 



Percentiles of Performance 

5th | 25th ■ 75th 95th 



95% Confidence Interval for Average (±2SE) 



o 

® 



Country average significantly higher 
than international average 

Country average significantly lower 
than international average 



Represents years of schooling counting from the first year of ISCED Level 1 . 

Taken from United Nations Development Programme's Human Development Report 2003, 
p. 237-240. 

Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

Nearly satisfied guidelines for sample participation rates only after replacement schools were 
included (see Exhibit C.2). 

Did not satisfy guidelines for sample participation rates (see Exhibit C.2). 



1 National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 

2 National Defined Population covers less than 90% of National Desired Population (see Exhibit C.1). 
m Korea tested the same cohort of students as other countries, but later in 2003, at the beginning of 

the next school year. 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 

A dash (-) indicates comparable data are not available. 
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Exhibit 2.3: Distribution of Mathematics Achievement for Applying Cognitive Domain 



MATHEMATICS 

Grade 




Years of Average 

Schooling* Age 



Mathematics Achievement Distribution 



Singapore 
t Hong Kong, SAR 
Japan 

Chinese Taipei 
Belgium (Flemish) 
Latvia 

Russian Federation 
1 Lithuania 
t Netherlands 
Hungary 
t England 
Cyprus 

Moldova, Rep. of 
t United States 

Italy 

t Australia 
t Scotland 
New Zealand 
Slovenia 
Armenia 
0 Norway 

Iran, Islamic Rep. of 
Philippines 
Morocco 
Tunisia 

Benchmarking Participants 
Indiana State, US 
Ontario Province, Can. 
Quebec Province, Can. 



4 

4 

4 

4 

4 

4 

3 or 4 
4 

4 

4 

5 
4 
4 
4 

4 

4 or 5 

5 

4.5 -5.5 
3 or 4 
4 
4 
4 
4 
4 
4 

4 

4 

4 



10.3 

10.2 

10.4 
10.2 
10.0 
11.1 
10.6 

10.9 

10.2 

10.5 

10.3 

9.9 
11.0 
10.2 

9.8 

9.9 

9.7 

10.0 

9.8 

10.9 

9.8 

10.4 

10.8 
11.0 

10.4 

10.5 
9.8 
10.1 




— 

mmmma 



Average 
Scale Score 


Human 

Development 

Index** 


595 (5.9) 


O 


0.884 


577 (3.3) 


O 


0.889 


566 (2.1) 


o 


0.932 


561 (1.9) 


o 


- 


546 (2.1) 


o 


0.937 


545 (3.3) 


o 


0.811 


542 (4.7) 


o 


0.779 


542 (2.9) 


o 


0.824 


541 (2.6) 


o 


0.938 


530 (3.4) 


o 


0.837 


526 (4.1) 


o 


0.930 


510 (2.8) 


o 


0.891 


507 (4.8) 


o 


0.700 


505 (2.6) 


o 


0.937 


495 (0.7) 


- 


494 (3.6) 




0.916 


490 (3.8) 




0.939 


487 (3.5) 




0.930 


486 (2.3) 




0.917 


477 (2.8) 


® 


0.881 


462 (3.2) 


® 


0.729 


446 (2.2) 


® 


0.944 


391 (3.8) 


® 


0.719 


364 (7.5) 


® 


0.751 


349 (4.5) 


® 


0.606 


348 (4.6) 


® 


0.740 


523 (3.3) 


o 


- 


498 (4.5) 




- 


498 (2.7) 




- 



Percentiles of Performance 

5th 25th ■ 75th | 95th 

95% Confidence Interval for Average (±2SE) 



O Country average significantly higher 
than international average 

® Country average significantly lower 
than international average 



Represents years of schooling counting from the first year of ISCED Level 1 . 

Taken from United Nations Development Programme's Human Development Report 2003, 
p. 237-240. 

Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 



0 Norway: 4 years of formal schooling, but First Grade is called "First grade/Preschool." 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 

A dash (-) indicates comparable data are not available. 
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Exhibit 2.4: Multiple Comparisons of Average Mathematics Achievement for Applying 

Cognitive Domain 



MATHEMATICS 

Grade 




Instructions: Read across the row for a country to compare performance with the countries listed along the top of the chart. The symbols indicate whether the average 
achievement of the country in the row is significantly lower than that of the comparison country, significantly higher than that of the comparison country, or if there is no 
statistically significant difference between the average achievement of the two countries. 



Countries 


Singapore 


Hong Kong, SAR 


Korea, Rep. of 


Chinese Taipei 


Japan 




Netherlands 


Belgium (Flemish) 


Estonia 


Hungary 


Malaysia 


Australia 


Scotland 


Sweden 


Latvia 


England 


Russian Federation 


Slovak Republic 


United States 


Lithuania 


New Zealand 


Israel 


Slovenia 


Italy 


Armenia 


Romania 


Bulgaria 


Norway 


Serbia 


Moldova, Rep. of 


Cyprus 


Singapore 




o 




° 


O 




o 


° 


° 


O 


O 




° 


° 


O 


O 


o 


o 


o 


O 


O 


° 


o 


° 


O 


o 


« 


° 


° 


o 


Hong Kong, SAR 




■ 




■ 


° 


O 


O 


O 


O 


O 


o 


O 


O 


O 


° 


O 


O 


O 


O 


O 


o 


O 


O 


O 


o 


o 


o 


O 


O 


o 


Korea, Rep. of 










o 


° 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


Chinese Taipei 


® 








o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


Japan 


® 


® 


® 


® 




° 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


o 


° 


° 


° 


o 


Netherlands 


® 


® 


® 


® 


® 






o 


o 


o 


o 


o 


o 


o 


° 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


Belgium (Flemish) 


® 


® 


® 


® 


® 








o 


o 


o 




° 




o 


o 


° 


° 


° 


o 


o 




° 




o 


o 


° 


° 


° 


o 


Estonia 


® 


® 


® 


® 


® 


® 






o 


o 


o 


o 


o 


O 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


Hungary 


® 


® 


® 


® 


® 


® 


® o 


o 








o 
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Note: 5% of these comparisons would be statistically significant by chance alone. 
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CHAPTER 2: MATHEMATICS ACHIEVEMENT IN THE COGNITIVE DOMAINS AT THE FOURTH AND EIGHTH GRADES 
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Exhibit 2.4: Multiple Comparisons of Average Mathematics Achievement for Applying 

Cognitive Domain 
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Instructions: Read across the row for a country to compare performance with the countries listed along the top of the chart. The 
symbols indicate whether the average achievement of the country in the row is significantly lower than that of the comparison country, 
significantly higher than that of the comparison country, or if there is no statistically significant difference between the average 
achievement of the two countries. 
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Note: 5% of these comparisons would be statistically significant by chance alone. 
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Exhibit 2.4: Multiple Comparisons of Average Mathematics Achievement for Applying 

Cognitive Domain 
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Instructions: Read across the row for a country to compare performance with the countries listed along the top of the chart. The symbols indicate 
whether the average achievement of the country in the row is significantly lower than that of the comparison country, significantly higher than 
that of the comparison country, or if there is no statistically significant difference between the average achievement of the two countries. 
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o 


o 


o 


o 


o 


o 


o 


Netherlands 


® 


® 


® 


® 












o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


Hungary 


® 


® 


® 


® 


® 


® 


® 


® 


® 






o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 




o 


o 


England 


® 


® 


® 


® 


® 


® 


® 


® 


® 






o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 




o 


o 


Cyprus 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 








o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


® 


o 


o 


Moldova, Rep. of 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 








o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


® 






United States 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 








o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


® 






Italy 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 










o 


o 


o 


o 


o 


o 


o 


® 






Australia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 










o 


o 


o 


o 


o 


o 


o 


® 






Scotland 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 










o 


o 


o 


o 


o 


o 


o 


® 




® 


New Zealand 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 










o 


o 


o 


o 


o 


o 


o 


® 


® 


® 


Slovenia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 




o 


o 


o 


o 


o 


o 


® 


® 


® 


Armenia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 




o 


o 


o 


o 


o 


® 


® 


® 


Norway 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 




o 


o 


o 


o 


® 


® 


® 


Iran, Islamic Rep. of 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 




o 


o 


o 


® 


® 


® 


Philippines 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 








® 


® 


® 


Morocco 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 








® 


® 


® 


Tunisia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 








® 


® 


® 


Benchmarking Participants 


























































Indiana State, US 


® 


® 


® 


® 


® 


® 


® 


® 


® 






o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


II 


o 


o 


Ontario Province, Can. 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 












o 


o 


o 


o 


o 


o 


o 


o 


® 






Quebec Province, Can. 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 










o 


o 


o 


o 


o 


o 


o 


o 


o 









O Average achievement 
significantly higher than 
comparison country 

® Average achievement 
significantly lower than 
comparison country 



Note: 5% of these comparisons would be statistically significant by chance alone. 
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CHAPTER 2: MATHEMATICS ACHIEVEMENT IN THE COGNITIVE DOMAINS AT THE FOURTH AND EIGHTH GRADES 



Exhibit 2.4 shows for the eighth (first two pages) and fourth 
(third page) grades how a country's average mathematics achievement 
in the applying domain compares to achievement in the other partici- 
pating countries. 

At the eighth grade, Singapore had the highest achievement 
in the applying domain. Hong Kong SAR, the Republic of Korea, and 
Chinese Taipei all performed equally well, but not as well as Singapore. 
These countries were followed by Japan with performance below that 
of the four top-scoring countries, but with significantly higher achieve- 
ment in this domain than all of the other participants at the eighth 
grade. The Netherlands and Belgium (Flemish) were outperformed by 
the five top-scoring Asian countries, but also did very well. 

At the fourth grade, results for the four Asian countries in the 
applying domain were nearly the same as for the knowing domain. 
The Singaporean students had the highest average achievement in 
the applying domain, followed by the students in Hong Kong SAR, 
who (with the exception of Singapore) had significantly higher 
achievement than students in the other participating countries. Japan 
and Chinese Taipei performed similarly to each other and had the 
next highest achievement after Hong Kong SAR. Compared to the 
knowing domain, however, several more countries performed simi- 
larly to each other and were in the second highest achieving group of 
countries. Belgium (Flemish), Latvia, the Russian Federation, Lithu- 
ania, and the Netherlands had significantly higher achievement in 
the applying domain than the rest of the participating countries and 
benchmarking entities. 

Reasoning 

The first and second pages of Exhibit 2.5 show the distribution of student 
mathematics achievement in the cognitive domain of reasoning at the 
eighth and fourth grades, respectively. Exhibit 2.6 shows, for the eighth 
and fourth grades (first two pages and third page, respectively), how a 
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TIMSS2003 



Exhibit 2.5: Distribution of Mathematics Achievement for Reasoning Cognitive Domain 



MATHEMATICS 

Grade 




Countries 


Years of 
Schooling* 


Average 

Age 


Singapore 


8 


14.3 


M Korea, Rep. of 


8 


14.6 


Chinese Taipei 


8 


14.2 


Japan 


8 


14.4 


t Hong Kong, SAR 


8 


14.4 


t Netherlands 


8 


14.3 


Belgium (Flemish) 


8 


14.1 


Hungary 


8 


14.5 


Estonia 


8 


15.2 


Australia 


8 or 9 


13.9 


t Scotland 


9 


13.7 


New Zealand 


8.5 -9.5 


14.1 


Sweden 


8 


14.9 


* United States 


8 


14.2 


Slovak Republic 


8 


14.3 


Malaysia 


8 


14.3 


Latvia 


8 


15.0 


Russian Federation 


7 or 8 


14.2 


Slovenia 


7 or 8 


13.8 


1 Lithuania 


8 


14.9 


Italy 


8 


13.9 


2 Israel 


8 


14.0 


Norway 


7 


13.8 


Bulgaria 


8 


14.9 


Armenia 


8 


14.9 


1 Serbia 


8 


14.9 


International Avg. 


8 


14.5 


Romania 


8 


15.0 


Cyprus 


8 


13.8 


Moldova, Rep. of 


8 


14.9 


2 Macedonia, Rep. of 


8 


14.6 


Jordan 


8 


13.9 


Bahrain 


8 


14.1 


Iran, Islamic Rep. of 


8 


14.4 


Lebanon 


8 


14.6 


Chile 


8 


14.2 


1 Indonesia 


8 


14.5 


Palestinian Nat’l Auth. 


8 


14.1 


Egypt 


8 


14.4 


Tunisia 


8 


14.8 


1 * Morocco 


8 


15.2 


Philippines 


8 


14.8 


Botswana 


8 


15.1 


Saudi Arabia 


8 


14.1 


Ghana 


8 


15.5 


South Africa 


8 


15.1 


* England 


9 


14.3 


Benchmarking Participants 


Basque Country, Spain 


8 


14.1 


Indiana State, US 


8 


14.5 


Ontario Province, Can. 


8 


13.8 


Quebec Province, Can. 


8 


14.2 




Average 
Scale Score 


Human 

Development 

Index** 


583 (3.5) 


O 


0.884 


582 (1.7) 


O 


0.879 


576 (4.2) 


o 


- 


576 (1.8) 


o 


0.932 


569 (3.1) 


o 


0.889 


541 (3.8) 


o 


0.938 


533 (2.8) 


o 


0.937 


529 (3.1) 


o 


0.837 


523 (3.0) 


o 


0.833 


515 (4.0) 


o 


0.939 


513 (3.4) 


o 


0.930 


509 (5.2) 


o 


0.917 


508 (3.3) 


o 


0.941 


505 (3.3) 


o 


0.937 


504 (3.2) 


o 


0.836 


503 (3.4) 


o 


0.790 


500 (3.4) 


o 


0.811 


496 (3.6) 


o 


0.779 


494 (2.5) 


o 


0.881 


489 (2.6) 


o 


0.824 


489 (2.9) 


o 


0.916 


483 (3.3) 


o 


0.905 


479 (2.8) 


o 


0.944 


471 (3.9) 




0.795 


468 (2.8) 




0.729 


468 (2.6) 




- 


467 (0.5) 


- 


458 (4.5) 


® 


0.773 


455 (1.7) 




0.891 


453 (4.0) 


® 


0.700 


438 (3.7) 


® 


0.784 


433 (3.7) 


® 


0.743 


424 (2.2) 


® 


0.839 


417 (2.8) 


® 


0.719 


410 (3.0) 


® 


0.752 


409 (3.5) 


® 


0.831 


406 (4.3) 


® 


0.682 


404 (2.7) 


® 


0.731 


400 (3.6) 


® 


0.648 


399 (2.7) 


® 


0.740 


391 (3.2) 


® 


0.606 


358 (5.8) 


® 


0.751 


353 (3.7) 


® 


0.614 


348 (4.3) 


® 


0.769 


313 (4.0) 


® 


0.567 


287 (5.0) 


® 


0.684 


509 (4.7) 


o 


0.930 




494 (2.4) 


o 


- 


503 (5.2) 


o 


- 


527 (3.0) 


o 


- 


539 (3.2) 


o 


- 



Percentiles of Performance 

5th 25th ■ 75th I 95th 

t 

95% Confidence Interval for Average (±2SE) 



o 

® 



Country average significantly higher 
than international average 

Country average significantly lower 
than international average 



Represents years of schooling counting from the first year of ISCED Level 1 . 

Taken from United Nations Development Programme's Human Development Report 2003, 
p. 237-240. 

Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

Nearly satisfied guidelines for sample participation rates only after replacement schools were 
included (see Exhibit C.2). 

Did not satisfy guidelines for sample participation rates (see Exhibit C.2). 

National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 



2 National Defined Population covers less than 90% of National Desired Population (see Exhibit C.1). 
m Korea tested the same cohort of students as other countries, but later in 2003, at the beginning of 
the next school year. 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 

A dash (-) indicates comparable data are not available. 
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Exhibit 2.5: Distribution of Mathematics Achievement for Reasoning Cognitive Domain 



MATHEMATICS 

Grade 



Countries 


Years of 
Schooling* 


Average 

Age 


Mathematics Achievement Distribution 


Singapore 


4 


10.3 


- 



t Hong Kong, SAR 


4 


10.2 


- 


Chinese Taipei 


4 


10.2 




Japan 


4 


10.4 




Belgium (Flemish) 


4 


10.0 


:■ 


t England 


5 


10.3 


■ — 


t Netherlands 


4 


10.2 




Latvia 


4 


11.1 


_ . _ 


1 Lithuania 


4 


10.9 


E — 


Russian Federation 


3 or 4 


10.6 


— — 


Hungary 


4 


10.5 


■ 


t United States 


4 


10.2 


:■ . — 


Cyprus 


4 


9.9 


■mnr—TirMi 


t Australia 


4 or 5 


9.9 


— ■ — 


New Zealand 


4.5 -5.5 


10.0 




Italy 


4 


9.8 


- 



Scotland 



9.7 



Average 
Scale Score 


Human 

Development 

Index** 


574 (6.1) 


O 


0.884 


564 (3.7) 


O 


0.889 


563 (2.2) 


o 


- 


562 (1.7) 


o 


0.932 


541 (2.2) 


o 


0.937 


537 (3.5) 


o 


0.930 


535 (2.9) 


o 


0.938 


531 (3.2) 


o 


0.811 


526 (3.1) 


o 


0.824 


526 (4.8) 


o 


0.779 


524 (3.2) 


o 


0.837 


519 (2.5) 


o 


0.937 


516 (2.4) 


o 


0.891 


507 (3.6) 


o 


0.939 


503 (2.2) 


o 


0.917 


499 (4.0) 




0.916 


498 (3.1) 




0.930 


495 (0.7) 


- 



Moldova, Rep. of 4 

Slovenia 3 or 4 

0 Norway 4 

Armenia 4 

Iran, Islamic Rep. of 4 

Morocco 4 

Philippines 4 

Tunisia 4 

Benchmarking Participants 

Indiana State, US 4 

Ontario Province, Can. 4 

Quebec Province, Can. 4 




494 (4.9) 




0.700 


485 (2.6) 


® 


0.881 


468 (2.1) 




0.944 


445 (3.1) 


® 


0.729 


400 (3.4) 


® 


0.719 


368 (4.4) 


® 


0.606 


359 (7.4) 


® 


0.751 


340 (4.2) 


® 


0.740 


528 (3.4) 


O 


- 


523 (3.6) 


O 


- 


512 (2.6) 


o 


- 



0 100 200 300 400 500 600 700 800 



Percentiles of Performance 

5th | 25th ■ 75th | 95th 

95% Confidence Interval for Average (±2SE) 



O Country average significantly higher 
than international average 

® Country average significantly lower 
than international average 



Represents years of schooling counting from the first year of ISCED Level 1 . 

Taken from United Nations Development Programme's Human Development Report 2003, 
p. 237-240. 

Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 



o Norway: 4 years of formal schooling, but First Grade is called "First grade/Preschool." 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 

some totals may appear inconsistent. 

A dash (-) indicates comparable data are not available. 
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Exhibit 2.6: Multiple Comparisons of Average Mathematics Achievement for Reasoning 

Cognitive Domain 



MATHEMATICS 

Grade 




Instructions: Read across the row for a country to compare performance with the countries listed along the top of the chart. The symbols indicate 
whether the average achievement of the country in the row is significantly lower than that of the comparison country, significantly higher than that 
of the comparison country, or if there is no statistically significant difference between the average achievement of the two countries. 



Countries 


Singapore 


Korea, Rep. of 


Chinese Taipei 


Japan 


Hong Kong, SAR 




Netherlands 


Belgium (Flemish) 


Hungary 


Estonia 


Australia 




Scotland 


New Zealand 


England 


Sweden 


United States 


Slovak Republic 


Malaysia 


Latvia 


Russian Federation 


Slovenia 


Lithuania 


Italy 


Israel 


Norway 


Bulgaria 




Armenia 


Serbia 


Romania 


Cyprus 


Moldova, Rep. of 


Singapore 






n 




O 


0 


0 


0 


0 


O 


0 


0 


0 


0 


O 


O 


0 


0 


0 


O 


O 


0 


0 


0 


O 


0 


0 


0 


0 


O 


Korea, Rep. of 




■ 


■ 


O 


O 


O 


O 


O 


O 


O 


O 


O 


O 


O 


O 


o 


O 


O 


O 


o 


o 


O 


O 


o 


o 


O 


O 


o 


o 


o 


Chinese Taipei 










0 


0 


0 


0 


o 


0 


0 


0 


0 


o 


o 


0 


0 


0 


o 


o 


0 


0 


0 


o 


0 


0 


0 


0 


o 


Japan 


® 


■ 


1 




o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


Hong Kong, SAR 


® 


® 






0 


0 


0 


0 


o 


0 


0 


0 


0 


o 


o 


0 


° 


° 


o 


o 


° 


° 


° 


o 


° 


° 


° 


° 


o 


Netherlands 


® 




® 


® 


® 






o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


Belgium (Flemish) 


® 


® 


3 


® 


® 








0 


o 


0 


0 


0 


0 


o 


o 


0 


0 


0 


o 


o 


0 


0 


0 


o 


0 


0 


0 


0 


o 


Hungary 


® 


® 


® 


® 


® 


® 








o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


o 


Estonia 


® 


® 


3 


® 


® 


® 


® 








0 


0 


0 


0 


o 


o 


0 


0 


0 


o 


o 


0 


0 


0 


o 


0 


0 


0 


0 


o 


Australia 


® 


® 


® 


® 


® 


® 


® 


® 


o 


o 


o 


0 


o 


o 


o 


0 


o 


o 


o 


0 


0 


o 


o 


Scotland 


® 


® 


3 


® 


® 


® 


® 


® 


® 














o 


0 


° 


0 


o 


o 


° 


0 


° 


o 


° 


0 


0 


° 


o 


New Zealand 


® 


® 


® 


® 


® 


® 


® 


® 


® 




















0 


o 


o 


o 


0 


o 


o 


o 


0 


0 


o 


o 


England 


® 


® 


3 


® 


® 


® 


® 


® 


® 


















0 


o 


o 


° 


0 


° 


o 


° 


0 


0 


° 


o 


Sweden 


® 


® 


® 


® 


® 


® 


® 


® 


® 


















0 


o 


o 


o 


0 


o 


o 


o 


0 


0 


o 


o 


United States 


® 


® 


3 


® 


® 


® 


® 


® 


® 














o 


o 


° 


0 


° 


o 


° 


0 


0 


° 


o 


Slovak Republic 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 










o 


o 


o 


0 


o 


o 


o 


0 


0 


o 


o 


Malaysia 


® 


® 


3 


® 


® 


® 


® 


® 


® 


® 


® 








m 


o 


® 


0 


® 


o 


® 


0 


0 


® 


o 


Latvia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


















o 


o 


0 


o 


o 


o 


0 


0 


O 


o 


Russian Federation 


® 


® 


3 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 








0 


° 


o 


° 


0 


0 


° 


o 


Slovenia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 








mm 


0 


o 


o 


o 


0 


0 


o 


o 


Lithuania 


® 


® 


3 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


®\ 


0 










° 


o 


° 


0 


0 


° 


o 


Italy 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


■a 




■ 


■ 


o 


0 


o 


0 


0 


o 


o 


Israel 


® 


® 


3 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


®l 


® 


® 


® 










o 


® 


0 


0 


® 


o 


Norway 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 






■1 


o 


0 


0 


o 


o 


Bulgaria 


® 


® 


3 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


®l 


® 


® 


\® 


®\ 


® 


0 










0 


® 


o 


Armenia 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 


® 








0 


o 


o 


Serbia 


® 


® 


3 


® 


® 


® 


® 


® 
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Note: 5% of these comparisons would be statistically significant by chance alone. 
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Exhibit 2.6: Multiple Comparisons of Average Mathematics Achievement for Reasoning 

Cognitive Domain 
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Instructions: Read across the row for a country to compare performance with the countries listed along the top of 
the chart. The symbols indicate whether the average achievement of the country in the row is significantly lower 
than that of the comparison country, significantly higher than that of the comparison country, or if there is no 
statistically significant difference between the average achievement of the two countries. 



Macedonia, Rep. of 


Jordan 


Bahrain 


Iran, Islamic Rep. of 


Lebanon 

Chile 


Indonesia 


Palestinian Nat'l Auth. 


Q. 

O) 

LU 


Tunisia 

Morocco 


Philippines 


Botswana 


Saudi Arabia 


Ghana 


South Africa 


Basque Country, Spain 


Indiana State, US 


Ontario Province, Can. 


Quebec Province, Can. 


Countries 




O 


o 


o 


o 


O O 








O O 










o 


O 






O 


Singapore 


O 


O 


O 


O 


o o 


o 


O 


o 


o o 


o 


o 


O 


o 


O 


O 


O 


O 


O 


Korea, Rep. of 


1 


o 


o 


o 




o o 








o o 






® 


® 


o 


o 


® 


® 


o 


Chinese Taipei 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 


o 


o 


o 


o 


Japan 


1 


o 


o 


o 




o o 








o o 






® 


® 


o 


o 


® 


® 


o 


Hong Kong, SAR 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 


o 


o 


o 




Netherlands 


1 


o 


o 


o 


o 


o o 








o o 






® 


® 


o 


o 


® 




Belgium (Flemish) 


o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 


o 


o 




® 


Hungary 


1 


o 


o 


o 




o o 








o o 






® 


® 


o 


o 


o 




® 


Estonia 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 


o 




® 


® 


Australia 


1 


o 


o 


o 


o 


o o 








o o 






® 


® 


o 


o 


® 


® 


Scotland 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


° 


o 


® 


® 


New Zealand 


1 


o 


o 


o 




o o 








o o 






® 


® 


o 


o 


® 


® 


England 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 


o 


® 


® 


Sweden 


1 


o 


o 


o 


o 


o o 








o o 






® 


® 


o 


o 


® 


® 


United States 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


° 


o 


® 


® 


Slovak Republic 


1 


o 


o 


o 




o o 








o o 






® 




o 


o 


® 


® 


Malaysia 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 




® 


® 


Latvia 


1 


o 


« 


o 


o 


o o 








o o 








® 


o 




® 


® 


Russian Federation 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 




® 


® 


Slovenia 


1 


o 


o 


o 




o o 








o o 






® 


® 


o 




« 


® 


® 


Lithuania 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 




® 


® 


® 


Italy 


1 


o 


o 


o 


o 


o o 








o o 






® 


® 


o 


® 


® 


® 


® 


Israel 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 


® 


® 


® 


® 


Norway 


1 


o 


o 


o 




o o 








o o 






® 


® 


o 


® 


® 


® 


® 


Bulgaria 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 


® 


® 


® 


® 


Armenia 


1 


o 


o 


o 


o 


o o 








o o 




® 


® 


® 


o 


® 


® 


® 


® 


Serbia 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 


® 


® 


® 


® 


Romania 


1 


o 


o 


o 




o o 








o o 






® 


® 


o 


® 


® 


® 


® 


Cyprus 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 




® 


® 


® 


® 


Moldova, Rep. of 


1 




o 




o o 








o o 




® 


® 




o 


® 


® 


® 


® 


Macedonia, Rep. of 




■1 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


O 


® 


® 


® 


® 


Jordan 


1 








o 


o o 








o o 




® 


® 


® 


o 


® 


® 


® 


® 


Bahrain 




® 


® 


® 




■■ 


o 


o 


o 


o o 


o 


o 


o 


o 


O 


® 


® 


® 


® 


Iran, Islamic Rep. of 


1 


® 


® 


® 




° 


o o 


° 


° 


° 


° 


o 


® 


® 


® 


® 


Lebanon 




® 


® 


® 


wmmm 




o o 


o 


o 


o 


o 


O 


® 


® 


® 


® 


Chile 


1 


® 


® 


® 


® 










o 


° 


° 


° 


° 


o 


® 


® 


® 


® 


Indonesia 




® 


® 


® 


® 


■■ 


■ 






o 


o 


o 


o 


o 


O 


® 


® 


® 


® 


Palestinian Nat'l Auth. 


1 


®l 


® 


0 


0 


® 










° 


° 


° 


° 


o 


® 


®l 


® 




Egypt 




® 


® 


® 


® 


® ® 










o 


o 


o 


o 




® 


® 


® 


® 


Tunisia 


1 


®[ 


®l 


0 


ra 


® ®l 


ra 


® 






° 


° 


° 


° 


o 


® 


®1 


[® 




Morocco 




® 


® 


® 


® 


® ® 


® 


® 


® 


® ® 








o 


m 


® 


® 


® 


® 


Philippines 


1 


®l 


® 


ra 


0 


® ® 


ra 




ra 


® ®1 








0 


o 


® 


® 


[® 




Botswana 




® 


® 


® 


® 


® ® 


® 


® 


® 


® ® 




■ 




0 


° 


® 


® 


® 


® 


Saudi Arabia 


1 


®j 


®l 


0 


0 


® "®j 


ra 




ra 


® _®j 


[®1 


® 


i® 




o 


® 


® 


r® 


[® 


Ghana 




® 


® 


® 


® 


® ® 


® 


® 


® 


® ® 


® 


® 


® 


® 


■■ 


® 


® 


® 


® 


South Africa 


1 


Benchmarking Participants 


O 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


o 






® 


® 


Basque Country, Spain 


1 


o 


0 


0 


0 


o o 


0 


0 


0 


o o 


0 


0 


0 


0 


0 








f® 


Indiana State, US 




o 


o 


o 


o 


o o 


o 


o 


o 


o o 


o 


o 


o 


o 


°u 


o 


o 




® 


Ontario Province, Can. 


1 


o 


° 


° 


° 


o o 


° 


° 


° 


o o 


° 


° 


° 


° 


o 


o 


° 


° 




Quebec Province, Can. 





O Average achievement 
significantly higher than 
comparison country 



(g Average achievement 
significantly lower than 
comparison country 



Note: 5% of these comparisons would be statistically significant by chance alone. 
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Exhibit 2.6: Multiple Comparisons of Average Mathematics Achievement for Reasoning 

Cognitive Domain 
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Instructions: Read across the row for a country to compare performance with the countries listed along the top of the chart. The symbols indicate 
whether the average achievement of the country in the row is significantly lower than that of the comparison country, significantly higher than 
that of the comparison country, or if there is no statistically significant difference between the average achievement of the two countries. 
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Note: 5% of these comparisons would be statistically significant by chance alone. 
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country's average mathematics achievement in the reasoning domain 
compares to achievement in the other participating countries. 

At the eighth grade, average achievement in the reasoning 
domain ranged from 583 in Singapore to 287 in South Africa. Twenty- 
four countries and the four benchmarking participants performed sig- 
nificantly above the international average, three countries (Bulgaria, 
Armenia, and Serbia) performed comparably to the international 
average, and 19 countries performed significantly below the average. 

At the eighth grade, looking at both Exhibits 2.5 and 2.6, it can 
be seen that the rank ordering of significant differences in achieve- 
ment is rather complicated for the reasoning domain. Singapore and the 
Republic of Korea had the highest average achievement in the reason- 
ing domain, nearly identical (583 and 582), but Singapore had a larger 
standard error (3.5 to 1.7). Thus, the Republic of Korea had significantly 
higher achievement than every participating country except Singapore 
and Chinese Taipei whereas Singapore (with the larger standard error) 
had higher average achievement than every participating country except 
the Republic of Korea, Chinese Taipei, and Japan. Chinese Taipei and 
Japan had the same average score (576) followed by Hong Kong SAR. 
Chinese Taipei (also with a relatively larger standard error of 4.2) did 
not perform statistically differently than the other three Asian countries, 
whereas a difference was found between the Republic of Korea and 
Japan due to their small standard errors. Hong Kong SAR was outper- 
formed only by Singapore and the Republic of Korea. The Netherlands 
and Belgium (Flemish) only were outperformed by the five top-scoring 
Asian countries. 

At the fourth grade, performance ranged from 574 for Singa- 
pore to 340 for Tunisia. Fifteen countries and the three benchmarking 
participants performed significantly above the international average, 
three countries (Italy, Scotland, and Moldova) performed essentially 
at the international average, and seven countries performed signifi- 
cantly below the international average. Singapore had the highest 
achievement, outperforming all countries except Hong Kong SAR and 
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Chinese Taipei. Hong Kong, Chinese Taipei and Japan had similar 
achievement followed by Belgium (Flemish), England, and the Neth- 
erlands (all with similar average achievement and only outperformed 
by the four highest-achieving Asian countries). 

Overview Across Domains 

At both the eighth and fourth grades, the countries with the highest 
achievement in each of the three cognitive domains also tended to be 
the highest-scoring countries (though not always in the same rank 
order) on the overall mathematics assessment. At the eighth grade (see 
Exhibit 1.1), the four countries with the highest overall mathematics 
achievement were Singapore followed by the Republic of Korea, Hong 
Kong SAR, and Chinese Taipei (only outperformed by Singapore). 
Japan had the next highest achievement outperforming all the rest of 
the participating countries except the previous four countries. Belgium 
(Flemish), the Netherlands, Estonia, Hungary, and the Canadian prov- 
ince of Quebec also performed well (at least as well or better than all 
other participants except the five Asian countries listed above). 

• In knowing, similar to overall mathematics achievement, the Repub- 
lic of Korea, Singapore, Hong Kong SAR, and Chinese Taipei had the 
highest achievement followed by Japan (see Exhibits 2.1 and 2.2). 
As a slight difference compared to the results for overall mathemat- 
ics achievement, the four top-scoring Asian countries performed 
similarly to each other in the knowing domain. Estonia, Belgium 
(Flemish), Hungary, and the Canadian province of Quebec were out- 
performed only by the five top-achieving Asian countries. 

• In applying, Singapore had the highest average achievement fol- 
lowed by Hong Kong SAR, the Republic of Korea, and Chinese Taipei 
and then Japan (see Exhibits 2.3 and 2.4). The Netherlands, Belgium 
(Flemish), and the Canadian province of Quebec were next (only 
outperformed by the five top-achieving Asian countries). 
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• In reasoning, Singapore and the Republic of Korea performed very 
similarly followed by Chinese Taipei and Japan and then Hong Kong 
SAR (see Exhibits 2.5 and 2.6). The Netherlands, Belgium (Flemish), 
and the Canadian province of Quebec also had relatively high 
achievement, only being outperformed by the five Asian countries. 

At the fourth grade, Singapore was the highest-performing 
country in overall mathematics followed by Hong Kong SAR, and then 
by Japan and Chinese Taipei who performed similarly (see Exhibit 1.1). 
Belgium (Flemish) had higher achievement than all countries except 
these four Asian countries. 

• In knowing, the pattern at the fourth grade was the same as for 
overall mathematics (see Exhibits 2.1 and 2.2). The four Asian coun- 
tries had the best achievement (Singapore followed by Hong Kong 
SAR, and then by Chinese Taipei and Japan) with Belgium (Flemish) 
having higher achievement than all countries except the four best- 
achieving Asian countries. 

• In applying, the pattern for the four high- achieving Asian countries 
was the same as for overall mathematics (see Exhibits 2.3 and 2.4). 
However, Belgium (Flemish), Latvia, the Russian Federation, Lithu- 
ania, and the Netherlands all followed, performing similarly to each 
other with lower achievement than the four Asian countries, but 
higher achievement than the rest of the participating countries. 

• In reasoning, Singapore, Hong Kong SAR, and Chinese Taipei had 
the highest achievement (see Exhibits 2.5 and 2.6). Japan had 
achievement similar to Hong Kong SAR and Chinese Taipei, but was 
outperformed by Singapore. Belgium (Flemish), England, and the 
Netherlands had achievement equal to or higher than all participants 
except the four top-achieving Asian countries. 

Just as countries with high achievement on the mathematics 
assessment as a whole had high achievement in the three cogni- 
tive domains, countries scoring lowest on the assessment as a whole 
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(e.g.. South Africa, Ghana, and Saudi Arabia at the eighth grade 
and the Philippines, Morocco, and Tunisia at the fourth grade) also 
tended to have low performance in all three cognitive domains. For 
some middle-performing countries, however, performance was more 
varied. For example, at the eighth grade, Armenia performed above 
the international average in the knowing and applying domains (480 
and 478) but essentially at the international average in the reasoning 
domain (468). 

Looking at the range in scale scores across the cognitive domains 
at the eighth grade, the differences in average achievement between the 
highest- and lowest-performing countries were largest in the knowing 
domain (360 score points), next largest in the applying domain (342), 
and smallest in the reasoning domain (296). As described in the fol- 
lowing sections, several more countries performed significantly above 
the international average in the knowing domain than in the applying 
and reasoning domains. 

At the fourth grade, with fewer countries, the range in perfor- 
mance between the highest- and lowest-performing countries was smaller 
than at the eighth grade, but the pattern was similar. The largest difference 
was in the knowing domain (288), next in the applying domain (246), 
and the smallest difference was in the reasoning domain (234). In each 
of the three cognitive domains, about the same number of participants 
performed above, similar to, or below the international average. 
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Chapter 3 

Achievement by Gender in the 
Mathematics Cognitive Domains at 
the Fourth and Eighth Grades 

This chapter presents average achievement by gender for the three 
mathematics cognitive domains. In general, as described in Chapter 1 in 
conjunction with Exhibit 1.1, on average, across the TIMSS 2003 par- 
ticipating countries and benchmarking entities, there was essentially no 
difference in achievement between boys and girls at either the eighth 
or fourth grade. Within the cognitive domains, however, there were 
significant differences by gender, especially at the eighth grade. 

At the eighth grade, girls had the advantage in more countries 
in the knowing domain of mathematics and, even more so in the rea- 
soning domain. Internationally across the TIMSS 2003 participants, 
girls had significantly higher achievement, on average, than boys in 
both these domains. Boys had the advantage in more countries in the 
applying domain. 

At the fourth grade, while performance was about the same 
internationally for boys and girls in the knowing domain, there was 
a significant difference, on average, favoring boys in the applying 
domain. Also, boys had significantly higher achievement in consider- 
ably more countries than did girls. In the reasoning domain, there was 
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essentially no difference internationally between boys and girls, but in 
the few countries where significant differences were found, girls had 
higher performance. 

Gender Differences in the Knowing Cognitive Domain 

Exhibit 3.1 shows gender differences in eighth-grade (first page) and 
fourth-grade (second page) mathematics achievement in the knowing 
domain. For each grade and for each country, it presents average 
achievement separately for girls and boys for each of the TIMSS 2003 
participants, as well as the difference between the means. Countries 
are shown in increasing order of this gender difference. The gender 
difference for each country is shown by a bar indicating the amount of 
the difference, whether the direction of the difference favored girls or 
boys, and whether the difference is statistically significant (indicated 
by a darkened bar). 

At the eighth grade, there was a small but significant difference 
favoring girls, on average, across countries in the knowing domain. 
Girls had significantly higher achievement than boys in the knowing 
domain in nearly half the countries (18 countries and the Basque 
Country, Spain). In contrast, boys had significantly higher achieve- 
ment than girls in four countries. 

At the fourth grade, there essentially was no difference inter- 
nationally in achievement in the knowing domain between boys and 
girls. There were differences in some countries, with girls outperform- 
ing boys in about the same number of countries as boys outperformed 
girls. Girls had significantly higher achievement than boys in four 
countries while boys had significantly higher achievement than girls 
in four countries and the two Canadian provinces. 
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TIMSS2003 



Exhibit 3.1: Average Mathematics Achievement by Gender for Knowing Cognitive Domain 



MATHEMATICS 

Grade 



Countries 


Girls 


Boys 


Difference 


Percent of 
Students 


Average 
Scale Score 


Percent of 
Students 


Average 
Scale Score 


(Absolute 

Value) 


Sweden 


51 (0.9) 


486 (2.5) 


49 (0.9) 


486 (2.3) 


0 (2.0) 


Japan 


49 (1.2) 


564 (3.8) 


51 (1.2) 


564 (3.4) 


1 (6.1) 


Hungary 


50 (1.0) 


537 (3.6) 


50 (1.0) 


536 (3.4) 


1 (3.2) 


Estonia 


50 (1.0) 


538 (3.2) 


50 (1.0) 


538 (3.0) 


1 (2.9) 


South Africa 


51 (0.9) 


260 (6.1) 


49 (0.9) 


261 (6.3) 


1 (5.8) 


New Zealand 


52 (1.7) 


484 (4.3) 


48 (1.7) 


486 (6.5) 


2 (5.3) 


Norway 


50 (0.8) 


451 (2.5) 


50 (0.8) 


449 (3.0) 


2 (3.5) 


1 Indonesia 


50 (0.7) 


423 (4.5) 


50 (0.7) 


421 (4.6) 


2 (3.4) 


Italy 


50 (0.9) 


483 (3.2) 


50 (0.9) 


485 (3.8) 


3 (2.8) 




50 (0.2) 


468 (0.6) 


50 (0.2) 


465 (0.6) 


3 (0.7) 


Bulgaria 


48 (1.3) 


487 (5.0) 


52 (1.3) 


484 (4.3) 


3 (4.5) 


Botswana 


51 (0.7) 


374 (3.0) 


49 (0.7) 


370 (3.6) 


4 (3.3) 


2 Israel 


52 (1.6) 


499 (3.3) 


48 (1.6) 


503 (4.0) 


4 (3.9) 


t Hong Kong, SAR 


50 (2.4) 


591 (3.7) 


50 (2.4) 


587 (4.6) 


4 (5.2) 


* United States 


52 (0.7) 


508 (3.0) 


48 (0.7) 


512 (3.0) 


4 (2.0) 


Egypt 


46 (2.7) 


413 (4.1) 


54 (2.7) 


409 (4.8) 


4 (6.0) 


t Netherlands 


49 (1.2) 


518 (3.5) 


51 (1.2) 


522 (3.6) 


4 (3.3) 


t Scotland 


50 (1.3) 


483 (3.9) 


50 (1.3) 


478 (3.4) 


4 (3.4) 


M Korea, Rep. of 


48 (2.8) 


589 (2.9) 


52 (2.8) 


594 (2.4) 


5 (3.2) 


Chinese Taipei 


48 (1.0) 


589 (5.0) 


52 (1.0) 


582 (5.0) 


7 (4.3) 


Belgium (Flemish) 


54 (2.1) 


534 (3.4) 


46 (2.1) 


541 (3.6) 


7 (4.8) 


Slovenia 


50 (0.9) 


502 (2.6) 


50 (0.9) 


495 (2.9) 


7 (3.5) 


Slovak Republic 


48 (1.3) 


521 (3.4) 


52 (1.3) 


514 (3.9) 


7 (3.2) 


Lebanon 


57 (1.8) 


444 (3.6) 


43 (1.8) 


452 (3.9) 


8 (4.0) 


1 Lithuania 


50 (0.9) 


514 (3.5) 


50 (0.9) 


507 (2.9) 


8 (3.3) 


Romania 


52 (0.9) 


489 (5.3) 


48 (0.9) 


481 (5.2) 


8 (3.6) 


1 * Morocco 


50 (1.8) 


383 (3.7) 


50 (1.8) 


392 (4.2) 


8 (5.4) 


Latvia 


49 (0.8) 


523 (2.9) 


51 (0.8) 


514 (3.4) 


10 (3.0) 


Singapore 


49 (0.8) 


596 (2.9) 


51 (0.8) 


586 (3.7) 


10 (2.6) 


Saudi Arabia 


43 (2.3) 


309 (6.9) 


57 (2.3) 


319 (6.2) 


10 (9.5) 


2 Macedonia, Rep. of 


49 (0.9) 


452 (4.1) 


51 (0.9) 


442 (4.2) 


10 (3.3) 


Russian Federation 


49 (1.2) 


525 (3.5) 


51 (1.2) 


514 (3.7) 


11 (2.5) 


Iran, Islamic Rep. of 


40 (4.1) 


412 (4.8) 


60 (4.1) 


401 (4.2) 


11 (7.3) 


Malaysia 


50 (1.8) 


511 (4.4) 


50 (1.8) 


501 (4.4) 


11 (4.2) 


Australia 


51 (2.2) 


491 (5.1) 


49 (2.2) 


502 (5.2) 


11 (6.4) 


Moldova, Rep. of 


51 (0.8) 


472 (4.9) 


49 (0.8) 


460 (4.6) 


12 (4.7) 


Armenia 


53 (0.7) 


486 (3.2) 


47 (0.7) 


474 (3.4) 


13 (3.2) 


1 Serbia 


49 (0.8) 


502 (3.1) 


51 (0.8) 


489 (3.1) 


13 (3.2) 


Philippines 


58 (0.9) 


395 (5.2) 


42 (0.9) 


379 (6.1) 


15 (4.4) 


Chile 


48 (1.6) 


378 (3.5) 


52 (1.6) 


393 (4.1) 


15 (4.5) 


Cyprus 


49 (0.6) 


474 (2.1) 


51 (0.6) 


458 (2.6) 


16 (2.6) 


Palestinian Nat'l Auth. 


55 (2.4) 


398 (4.8) 


45 (2.4) 


382 (5.6) 


17 (7.3) 


Tunisia 


53 (0.7) 


388 (3.6) 


47 (0.7) 


410 (3.0) 


22 (2.8) 


Ghana 


45 (0.9) 


220 (7.0) 


55 (0.9) 


243 (6.4) 


23 (6.1) 


Jordan 


49 (1.7) 


442 (5.5) 


51 (1.7) 


415 (6.2) 


26 (7.2) 


Bahrain 


50 (0.4) 


419 (2.8) 


50 (0.4) 


383 (3.2) 


36 (4.0) 


* England 


50 (2.4) 


488 (4.5) 


50 (2.4) 


489 (5.2) 


1 (5.6) 


Benchmarking Participants 


Basque Country, Spain 


49 (1.7) 


498 (2.2) 


51 (1.7) 


492 (3.0) 


7 (2.8) 


Indiana State, US 


49 (1.2) 


512 (4.7) 


51 (1.2) 


518 (5.2) 


6 (3.5) 


Ontario Province, Can. 


51 (0.9) 


512 (2.9) 


49 (0.9) 


513 (3.1) 


1 (2.9) 


Quebec Province, Can. 


50 (1.6) 


535 (3.3) 


50 (1.6) 


540 (3.1) 


5 (3.4) 




Gender Difference 

Girls Boys 

Scored Higher Scored Higher 







+ Met guidelines for sample participation rates only after replacement schools were included (see 1 

Exhibit C.2). 2 

t Nearly satisfied guidelines for sample participation rates only after replacement schools were ^ 

included (see Exhibit C.2). 

t Did not satisfy guidelines for sample participation rates (see Exhibit C.2). / 



■ Gender difference statistically significant 
□ Gender difference not statistically significant 

National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 
National Defined Population covers less than 90% of National Desired Population (see Exhibit C.1). 
Korea tested the same cohort of students as other countries, but later in 2003, at the beginning of the 
next school year. 

Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 
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TIMSS2003 



Exhibit 3.1: Average Mathematics Achievement by Gender for Knowing Cognitive Domain 



MATHEMATICS 

Grade 



Countries 


Girls 


Boys 


Difference 


Percent of 
Students 


Average 
Scale Score 


Percent of 
Students 


Average 
Scale Score 


(Absolute 

Value) 


t England 


50 (0.9) 


534 (4.4) 


50 (0.9) 


534 (5.3) 


0 (3.7) 




49 (0.2) 


496 (0.9) 


51 (0.2) 


495 (0.9) 


0 (0.9) 


Tunisia 


48 (0.9) 


337 (4.8) 


52 (0.9) 


338 (4.6) 


1 (3.8) 


t Australia 


50 (1.0) 


501 (4.1) 


50 (1 .0) 


502 (4.6) 


1 (4.1) 


t Hong Kong, SAR 


47 (1.1) 


574 (3.9) 


53 (1.1) 


573 (3.6) 


1 (3.5) 


Russian Federation 


50 (0.7) 


513 (5.8) 


50 (0.7) 


514 (5.8) 


1 (4.6) 


Japan 


49 (0.6) 


565 (2.6) 


51 (0.6) 


564 (3.0) 


2 (3.7) 


1 Lithuania 


49 (0.9) 


522 (3.3) 


51 (0.9) 


520 (3.6) 


2 (3.6) 


Chinese Taipei 


48 (0.5) 


564 (2.4) 


52 (0.5) 


566 (2.6) 


2 (2.3) 


New Zealand 


50 (1.1) 


494 (2.9) 


50 (1.1) 


492 (2.5) 


2 (3.2) 


Hungary 


50 (0.9) 


516 (4.1) 


50 (0.9) 


518 (3.6) 


2 (4.0) 


Norway 


50 (0.8) 


447 (2.5) 


50 (0.8) 


449 (3.1) 


2 (3.7) 


Slovenia 


48 (1.1) 


469 (3.1) 


52 (1.1) 


471 (3.3) 


2 (3.7) 


Latvia 


49 (0.9) 


518 (3.1) 


51 (0.9) 


515 (3.5) 


3 (3.5) 


Belgium (Flemish) 


50 (1.0) 


556 (2.6) 


50 (1 .0) 


560 (2.9) 


4 (3.5) 


t Netherlands 


49 (1.1) 


527 (3.0) 


51 (1.1) 


533 (2.6) 


6 (3.5) 


Morocco 


49 (1.1) 


356 (5.6) 


51 (1.1) 


363 (4.3) 


7 (4.7) 


t United States 


50 (0.5) 


525 (2.4) 


50 (0.5) 


532 (3.0) 


7 (2.3) 


Italy 


48 (0.8) 


510 (4.3) 


52 (0.8) 


518 (4.1) 


8 (3.0) 


Cyprus 


49 (0.7) 


496 (3.1) 


51 (0.7) 


504 (3.6) 


8 (3.6) 


Philippines 


51 (1.0) 


389 (8.0) 


49 (1 .0) 


380 (6.3) 


9 (4.4) 


t Scotland 


51 (1.0) 


478 (3.1) 


49 (1 .0) 


489 (4.2) 


11 (4.2) 


Singapore 


49 (1.4) 


632 (6.4) 


51 (1.4) 


620 (7.2) 


11 (4.5) 


Moldova, Rep. of 


50 (0.8) 


507 (5.9) 


50 (0.8) 


495 (5.3) 


12 (4.1) 


Iran, Islamic Rep. of 


39 (4.2) 


411 (6.6) 


61 (4.2) 


399 (4.5) 


12 (7.7) 


Armenia 


49 (0.8) 


455 (3.7) 


51 (0.8) 


439 (4.3) 


16 (3.3) 


Benchmarking Participants 












Indiana State, US 


52 (1.1) 


541 (4.4) 


48 (1.1) 


546 (4.3) 


4 (4.5) 


Ontario Province, Can. 


48 (1.1) 


508 (4.4) 


52 (1.1) 


519 (5.7) 


11 (5.5) 


Quebec Province, Can. 


50 (0.9) 


501 (3.2) 


50 (0.9) 


508 (2.9) 


7 (2.7) 



Gender Difference 



Girls 

Scored Higher 



Boys 

Scored Higher 




■ Gender difference statistically significant 
□ Gender difference not statistically significant 



t Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

1 National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 



( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 
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Gender Differences in the Applying Cognitive Domain 

Exhibit 3.2 shows achievement differences between girls and boys for 
the applying domain for the eighth and fourth grades, (on the first 
and second pages, respectively). For the applying domain at the eighth 
grade, boys had significantly higher achievement in more countries 
than girls. Girls had significantly higher achievement than boys in 
seven countries, and boys had significantly higher achievement than 
girls in 13 countries and two benchmarking participants (the US state 
of Indiana and the Canadian province of Quebec). 

Fourth grade had a corresponding pattern for the applying 
domain, with boys having significantly higher achievement in more 
countries than girls. Girls had higher achievement in the applying 
domain in four countries whereas boys had higher achievement in 
seven countries and the two Canadian provinces. Also, internationally, 
on average, there was a small but significant difference favoring boys. 
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TIMSS2003 



Exhibit 3.2: Average Mathematics Achievement by Gender for Applying Cognitive Domain 



MATHEMATICS 

Grade 



o 

O 



Countries 


Girls 


Boys 


Difference 


Percent of 
Students 


Average 
Scale Score 


Percent of 
Students 


Average 
Scale Score 


(Absolute 

Value) 


Norway 


50 (0.8) 


469 (2.8) 


50 (0.8) 


468 (3.4) 


0 (3.2) 


Slovenia 


50 (0.9) 


491 (3.0) 


50 (0.9) 


491 (2.8) 


0 (3.6) 


Russian Federation 


49 (1.2) 


503 (3.8) 


51 (1.2) 


503 (4.1) 


0 (2.6) 


t Hong Kong, SAR 


50 (2.4) 


584 (3.7) 


50 (2.4) 


584 (4.5) 


1 (5.1) 


Romania 


52 (0.9) 


475 (5.4) 


48 (0.9) 


474 (5.3) 


1 (3.9) 


1 Indonesia 


50 (0.7) 


408 (5.0) 


50 (0.7) 


409 (5.3) 


1 (3.3) 


New Zealand 


52 (1.7) 


496 (4.7) 


48 (1 .7) 


497 (7.2) 


1 (5.9) 


Bulgaria 


48 (1.3) 


471 (6.0) 


52 (1.3) 


472 (4.9) 


1 (5.5) 




50 (0.2) 


466 (0.6) 


50 (0.2) 


467 (0.6) 


1 (0.6) 


Sweden 


51 (0.9) 


504 (3.2) 


49 (0.9) 


506 (2.8) 


1 (2.2) 


Palestinian Nat’l Auth. 


55 (2.4) 


389 (4.1) 


45 (2.4) 


388 (4.6) 


1 (5.8) 


Latvia 


49 (0.8) 


505 (3.5) 


51 (0.8) 


504 (4.1) 


2 (3.4) 


1 Lithuania 


50 (0.9) 


499 (3.2) 


50 (0.9) 


497 (3.3) 


2 (2.9) 


Botswana 


51 (0.7) 


370 (3.0) 


49 (0.7) 


368 (2.9) 


2 (2.4) 


Japan 


49 (1.2) 


563 (4.4) 


51 (1.2) 


565 (3.6) 


2 (6.7) 


t Scotland 


50 (1.3) 


506 (4.8) 


50 (1.3) 


504 (3.8) 


3 (3.8) 


1 Serbia 


49 (0.8) 


468 (3.5) 


51 (0.8) 


466 (3.1) 


3 (2.9) 


South Africa 


51 (0.9) 


267 (5.9) 


49 (0.9) 


271 (6.5) 


3 (6.1) 


Estonia 


50 (1.0) 


531 (3.3) 


50 (1 .0) 


526 (3.2) 


4 (2.9) 


Chinese Taipei 


48 (1.0) 


584 (5.1) 


52 (1.0) 


580 (5.1) 


4 (4.2) 


Egypt 


46 (2.7) 


401 (4.3) 


54 (2.7) 


406 (4.9) 


5 (6.3) 


2 Macedonia, Rep. of 


49 (0.9) 


431 (4.2) 


51 (0.9) 


426 (4.3) 


6 (3.9) 


Slovak Republic 


48 (1.3) 


499 (4.0) 


52 (1.3) 


505 (4.3) 


6 (3.6) 


M Korea, Rep. of 


48 (2.8) 


581 (2.9) 


52 (2.8) 


587 (2.3) 


6 (2.9) 


Malaysia 


50 (1.8) 


515 (5.1) 


50 (1 .8) 


508 (4.8) 


7 (4.6) 


Iran, Islamic Rep. of 


40 (4.1) 


420 (4.6) 


60 (4.1) 


413 (4.1) 


7 (7.2) 


Italy 


50 (0.9) 


479 (3.0) 


50 (0.9) 


488 (4.0) 


8 (3.0) 


Armenia 


53 (0.7) 


482 (3.5) 


47 (0.7) 


473 (3.5) 


8 (3.6) 


* United States 


52 (0.7) 


497 (3.5) 


48 (0.7) 


506 (3.5) 


9 (2.1) 


Moldova, Rep. of 


51 (0.8) 


462 (4.0) 


49 (0.8) 


453 (4.5) 


9 (3.3) 


Lebanon 


57 (1.8) 


422 (3.7) 


43 (1.8) 


432 (4.2) 


10 (4.0) 


t Netherlands 


49 (1.2) 


538 (4.0) 


51 (1.2) 


548 (4.3) 


10 (3.8) 


Philippines 


58 (0.9) 


383 (4.8) 


42 (0.9) 


373 (5.5) 


10 (3.5) 


2 Israel 


52 (1.6) 


490 (3.7) 


48 (1 .6) 


500 (4.6) 


10 (4.2) 


Singapore 


49 (0.8) 


617 (3.6) 


51 (0.8) 


606 (4.1) 


11 (3.1) 


Hungary 


50 (1.0) 


517 (3.8) 


50 (1 .0) 


529 (4.0) 


11 (3.5) 


Saudi Arabia 


43 (2.3) 


332 (6.1) 


57 (2.3) 


344 (4.5) 


12 (7.9) 


Ghana 


45 (0.9) 


286 (4.9) 


55 (0.9) 


299 (4.8) 


13 (5.2) 


Australia 


51 (2.2) 


501 (6.1) 


49 (2.2) 


516 (6.0) 


15 (7.5) 


Belgium (Flemish) 


54 (2.1) 


529 (3.3) 


46 (2.1) 


544 (3.7) 


15 (4.6) 


Cyprus 


49 (0.6) 


465 (1 .9) 


51 (0.6) 


450 (2.5) 


16 (3.1) 


1 * Morocco 


50 (1.8) 


377 (3.4) 


50 (1 .8) 


393 (3.3) 


16 (3.4) 


Chile 


48 (1.6) 


382 (3.6) 


52 (1.6) 


399 (4.2) 


18 (4.6) 


Tunisia 


53 (0.7) 


407 (2.6) 


47 (0.7) 


433 (2.4) 


26 (2.1) 


Jordan 


49 (1.7) 


436 (4.9) 


51 (1.7) 


409 (5.8) 


27 (6.9) 


Bahrain 


50 (0.4) 


411 (2.3) 


50 (0.4) 


384 (2.3) 


27 (3.2) 


* England 


50 (2.4) 


503 (5.4) 


50 (2.4) 


504 (6.0) 


1 (6.3) 


Benchmarking Participants 


Basque Country, Spain 


49 (1.7) 


483 (2.5) 


51 (1.7) 


480 (3.2) 


3 (3.5) 


Indiana State, US 


49 (1.2) 


498 (5.7) 


51 (1.2) 


516 (6.7) 


17 (4.4) 


Ontario Province, Can. 


51 (0.9) 


520 (3.3) 


49 (0.9) 


525 (3.5) 


5 (3.1) 


Quebec Province, Can. 


50 (1.6) 


539 (3.6) 


50 (1 .6) 


549 (3.4) 


10 (3.4) 



Gender Difference 


Girls 


Boys 


Scored Higher 


Scored Higher 
















I 



I 











t Met guidelines for sample participation rates only after replacement schools were included (see 1 

Exhibit C.2). 2 

$ Nearly satisfied guidelines for sample participation rates only after replacement schools were ^ 

included (see Exhibit C.2). 

I Did not satisfy guidelines for sample participation rates (see Exhibit C.2). / 



■ Gender difference statistically significant 
□ Gender difference not statistically significant 

National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 
National Defined Population covers less than 90% of National Desired Population (see Exhibit C.1). 
Korea tested the same cohort of students as other countries, but later in 2003, at the beginning of 
the next school year. 

Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 
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Exhibit 3.2: Average Mathematics Achievement by Gender for Applying Cognitive Domain 



MATHEMATICS 

Grade 



Countries 


Girls 


Boys 


Difference 


Percent of 
Students 


Average 
Scale Score 


Percent of 
Students 


Average 
Scale Score 


(Absolute 

Value) 


Iran, Islamic Rep. of 


39 (4.2) 


391 (6.1) 


61 (4.2) 


391 (4.8) 


0 (7.9) 


t Hong Kong, SAR 


47 (1.1) 


576 (3.5) 


53 (1.1) 


577 (3.5) 


0 (2.6) 


Chinese Taipei 


48 (0.5) 


561 (2.0) 


52 (0.5) 


562 (2.2) 


1 (2.0) 


Latvia 


49 (0.9) 


545 (3.4) 


51 (0.9) 


546 (4.0) 


1 (3.4) 


New Zealand 


50 (1.1) 


485 (3.0) 


50 (1.1) 


486 (2.5) 


1 (3.2) 


Hungary 


50 (0.9) 


530 (4.0) 


50 (0.9) 


531 (3.7) 


2 (3.7) 


International Avg. 


49 (0.2) 


494 (0.8) 


51 (0.2) 


497 (0.8) 


2 (0.8) 


t England 


50 (0.9) 


524 (4.1) 


50 (0.9) 


528 (4.9) 


4 (4.0) 


Belgium (Flemish) 


50 (1.0) 


544 (2.5) 


50 (1.0) 


548 (2.7) 


4 (3.0) 


1 Lithuania 


49 (0.9) 


541 (3.7) 


51 (0.9) 


545 (3.7) 


4 (3.7) 


t Australia 


50 (1.0) 


487 (4.3) 


50 (1.0) 


492 (4.4) 


5 (3.9) 


Japan 


49 (0.6) 


563 (2.6) 


51 (0.6) 


569 (2.3) 


5 (2.5) 


Tunisia 


48 (0.9) 


351 (5.1) 


52 (0.9) 


346 (4.7) 


6 (3.6) 


Russian Federation 


50 (0.7) 


539 (5.0) 


50 (0.7) 


545 (5.1) 


6 (3.8) 


Norway 


50 (0.8) 


443 (2.8) 


50 (0.8) 


449 (3.0) 


6 (3.8) 


Armenia 


49 (0.8) 


465 (3.2) 


51 (0.8) 


459 (3.7) 


6 (2.8) 


t Netherlands 


49 (1.1) 


538 (2.8) 


51 (1.1) 


545 (3.2) 


7 (3.1) 


Morocco 


49 (1.1) 


345 (5.6) 


51 (1.1) 


352 (4.4) 


7 (4.5) 


Slovenia 


48 (1.1) 


474 (3.1) 


52 (1.1) 


481 (3.8) 


7 (4.2) 


Italy 


48 (0.8) 


489 (4.3) 


52 (0.8) 


498 (3.5) 


8 (3.0) 


Singapore 


49 (1.4) 


599 (5.8) 


51 (1.4) 


590 (6.6) 


9 (4.1) 


Moldova, Rep. of 


50 (0.8) 


511 (5.3) 


50 (0.8) 


502 (5.0) 


9 (3.6) 


t United States 


50 (0.5) 


501 (2.8) 


50 (0.5) 


510 (2.9) 


9 (2.1) 


t Scotland 


51 (1.0) 


482 (3.6) 


49 (1.0) 


492 (4.6) 


11 (4.3) 


Cyprus 


49 (0.7) 


504 (3.1) 


51 (0.7) 


516 (2.9) 


12 (2.4) 


Philippines 


51 (1.0) 


370 (8.8) 


49 (1.0) 


357 (6.7) 


13 (4.8) 


ichmarking Participants 












Indiana State, US 


52 (1.1) 


521 (3.4) 


48 (1.1) 


525 (4.2) 


4 (3.8) 


Ontario Province, Can. 


48 (1.1) 


491 (3.8) 


52 (1.1) 


505 (5.8) 


14 (4.4) 


Quebec Province, Can. 


50 (0.9) 


493 (3.2) 


50 (0.9) 


503 (2.9) 


9 (2.8) 



Gender Difference 



Boys 

Scored Higher 




■ Gender difference statistically significant 
□ Gender difference not statistically significant 



t Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

1 National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 



( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 
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CHAPTER 3: ACHIEVEMENT BY GENDER IN THE MATHEMATICS COGNITIVE DOMAINS AT THE FOURTH AND EIGHTH GRADES 



Gender Differences in the Reasoning Cognitive Domain 

Exhibit 3.3 shows gender achievement differences in the reason- 
ing domain at the eighth grade (first page) and fourth grade (second 
page). On average, across all countries, eighth-grade girls had signifi- 
cantly higher achievement than boys in the reasoning domain. In this 
domain, girls had signih candy higher achievement than boys in 1 7 coun- 
tries and the Basque Country, Spain whereas boys had higher achieve- 
ment in only two countries (Morocco and Tunisia). 

At the fourth grade this pattern was similar, but far less pronounced. 
There was essentially no difference in achievement internationally between 
fourth-grade boys and girls in the reasoning domain. However, girls had 
higher achievement than boys in three countries whereas boys did not 
outperform girls in any country or benchmarking entity. 
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Exhibit 3.3: Average Mathematics Achievement by Gender for Reasoning 

Cognitive Domain 



MATHEMATICS 

Grade 




Countries 


Girls 


Boys 


Difference 


Percent of 
Students 


Average 
Scale Score 


Percent of 
Students 


Average 
Scale Score 


(Absolute 

Value) 


* United States 


52 (0.7) 


505 (3.3) 


48 (0.7) 


506 (3.7) 


0 (2.4) 


2 Israel 


52 (1.6) 


483 (3.4) 


48 (1.6) 


483 (4.6) 


0 (4.6) 


Australia 


51 (2.2) 


515 (5.1) 


49 (2.2) 


516 (5.1) 


1 (6.2) 


Bulgaria 


48 (1.3) 


471 (5.2) 


52 (1.3) 


471 (4.4) 


1 (5.7) 


South Africa 


51 (0.9) 


287 (5.6) 


49 (0.9) 


286 (5.7) 


1 (5.1) 


1 Indonesia 


50 (0.7) 


405 (4.4) 


50 (0.7) 


406 (4.8) 


1 (3.0) 


Japan 


49 (1.2) 


575 (3.7) 


51 (1.2) 


576 (3.0) 


1 (5.6) 


t Netherlands 


49 (1.2) 


540 (4.3) 


51 (1.2) 


542 (4.5) 


1 (4.2) 


Hungary 


50 (1.0) 


530 (3.7) 


50 (1.0) 


528 (3.5) 


2 (3.6) 


Saudi Arabia 


43 (2.3) 


347 (5.7) 


57 (2.3) 


349 (6.1) 


2 (8.4) 


Slovak Republic 


48 (1.3) 


505 (3.3) 


52 (1.3) 


503 (4.2) 


2 (4.2) 


t Hong Kong, SAR 


50 (2.4) 


571 (3.5) 


50 (2.4) 


567 (4.4) 


3 (5.0) 


Egypt 


46 (2.7) 


402 (4.5) 


54 (2.7) 


399 (5.1) 


3 (6.5) 


Russian Federation 


49 (1.2) 


498 (4.0) 


51 (1.2) 


494 (3.8) 


3 (3.1) 


Romania 


52 (0.9) 


460 (5.0) 


48 (0.9) 


456 (5.0) 


4 (4.6) 


International Avg. 


50 (0.2) 


469 (0.5) 


50 (0.2) 


465 (0.6) 


4 (0.6) 


Malaysia 


50 (1.8) 


505 (3.9) 


50 (1.8) 


501 (3.9) 


4 (3.9) 


Italy 


50 (0.9) 


486 (3.0) 


50 (0.9) 


491 (3.4) 


5 (2.7) 


M Korea, Rep. of 


48 (2.8) 


580 (2.4) 


52 (2.8) 


584 (2.1) 


5 (2.8) 


Belgium (Flemish) 


54 (2.1) 


531 (3.8) 


46 (2.1) 


536 (3.6) 


5 (4.8) 


Sweden 


51 (0.9) 


511 (4.1) 


49 (0.9) 


505 (3.3) 


5 (3.6) 


Botswana 


51 (0.7) 


356 (3.5) 


49 (0.7) 


351 (4.5) 


6 (3.0) 


Lebanon 


57 (1.8) 


407 (3.5) 


43 (1.8) 


413 (4.9) 


6 (5.9) 


Chile 


48 (1.6) 


406 (4.1) 


52 (1.6) 


412 (4.2) 


6 (4.5) 


Estonia 


50 (1.0) 


526 (3.4) 


50 (1.0) 


519 (3.4) 


7 (3.2) 


1 Serbia 


49 (0.8) 


472 (3.3) 


51 (0.8) 


464 (2.8) 


7 (3.2) 


Latvia 


49 (0.8) 


504 (3.6) 


51 (0.8) 


496 (4.4) 


8 (4.2) 


1 Lithuania 


50 (0.9) 


492 (3.0) 


50 (0.9) 


484 (3.3) 


8 (2.9) 


t Scotland 


50 (1.3) 


517 (4.3) 


50 (1.3) 


509 (3.4) 


8 (3.7) 


Ghana 


45 (0.9) 


309 (4.6) 


55 (0.9) 


317 (5.0) 


8 (5.2) 


Chinese Taipei 


48 (1.0) 


581 (4.3) 


52 (1.0) 


572 (4.8) 


9 (3.8) 


Armenia 


53 (0.7) 


473 (3.4) 


47 (0.7) 


463 (4.3) 


9 (5.3) 


Moldova, Rep. of 


51 (0.8) 


458 (4.2) 


49 (0.8) 


448 (4.6) 


10 (3.7) 


Iran, Islamic Rep. of 


40 (4.1) 


423 (3.8) 


60 (4.1) 


413 (4.4) 


10 (6.2) 


Singapore 


49 (0.8) 


589 (3.3) 


51 (0.8) 


579 (4.4) 


10 (3.5) 


1 * Morocco 


50 (1.8) 


387 (3.9) 


50 (1.8) 


397 (4.0) 


11 (4.4) 


Slovenia 


50 (0.9) 


500 (3.1) 


50 (0.9) 


488 (3.2) 


12 (3.9) 


Philippines 


58 (0.9) 


363 (5.9) 


42 (0.9) 


350 (6.4) 


13 (4.1) 


2 Macedonia, Rep. of 


49 (0.9) 


444 (4.1) 


51 (0.9) 


432 (4.7) 


13 (4.7) 


Palestinian Nat'l Auth. 


55 (2.4) 


410 (3.8) 


45 (2.4) 


397 (4.2) 


14 (6.0) 


Norway 


50 (0.8) 


486 (3.1) 


50 (0.8) 


472 (3.5) 


14 (3.3) 


Jordan 


49 (1.7) 


442 (4.1) 


51 (1.7) 


425 (5.3) 


18 (6.2) 


New Zealand 


52 (1.7) 


519 (5.4) 


48 (1.7) 


499 (6.7) 


19 (6.2) 


Cyprus 


49 (0.6) 


465 (2.3) 


51 (0.6) 


446 (2.4) 


20 (3.3) 


Tunisia 


53 (0.7) 


390 (3.3) 


47 (0.7) 


410 (3.3) 


20 (3.8) 


Bahrain 


50 (0.4) 


435 (2.5) 


50 (0.4) 


412 (3.2) 


23 (3.8) 


* England 


50 (2.4) 


513 (4.8) 


50 (2.4) 


506 (5.9) 


8 (5.4) 


Benchmarking Participants 


Basque Country, Spain 


49 (1.7) 


500 (2.8) 


51 (1.7) 


488 (3.8) 


13 (4.6) 


Indiana State, US 


49 (1.2) 


502 (4.6) 


51 (1.2) 


503 (6.5) 


1 (4.6) 


Ontario Province, Can. 


51 (0.9) 


527 (3.8) 


49 (0.9) 


528 (3.6) 


1 (4.3) 


Quebec Province, Can. 


50 (1.6) 


537 (3.7) 


50 (1.6) 


540 (3.9) 


2 (4.2) 





t Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

t Nearly satisfied guidelines for sample participation rates only after replacement schools were 
included (see Exhibit C.2). 

i Did not satisfy guidelines for sample participation rates (see Exhibit C.2). 



■ Gender difference statistically significant 
□ Gender difference not statistically significant 

1 National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 

2 National Defined Population covers less than 90% of National Desired Population (see Exhibit C.1). 
m Korea tested the same cohort of students as other countries, but later in 2003, at the beginning of the 

next school year. 

( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 
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Exhibit 3.3: Average Mathematics Achievement by Gender for Reasoning 

Cognitive Domain 



MATHEMATICS 

Grade 





Girls 


Boys 


Difference 


Countries 


Percent of 
Students 


Average 
Scale Score 


Percent of 
Students 


Average 
Scale Score 


(Absolute 

Value) 


Latvia 


49 (0.9) 


531 (3.3) 


51 (0.9) 


531 (4.1) 


0 (3.9) 


Belgium (Flemish) 


50 (1.0) 


541 (2.6) 


50 (1 .0) 


541 (2.8) 


0 (3.1) 


t Australia 


50 (1.0) 


507 (3.9) 


50 (1 .0) 


507 (4.2) 


0 (3.4) 




49 (0.2) 


496 (0.9) 


51 (0.2) 


495 (0.8) 


1 (0.9) 


Slovenia 


48 (1.1) 


486 (3.0) 


52 (1.1) 


485 (3.6) 


1 (4.2) 


Hungary 


50 (0.9) 


525 (4.0) 


50 (0.9) 


524 (3.8) 


1 (4.4) 


Tunisia 


48 (0.9) 


340 (5.8) 


52 (0.9) 


339 (4.7) 


1 (6.3) 


t Hong Kong, SAR 


47 (1.1) 


565 (4.0) 


53 (1.1) 


563 (3.8) 


2 (2.7) 


New Zealand 


50 (1.1) 


502 (2.9) 


50 (1.1) 


504 (2.4) 


2 (3.1) 


Cyprus 


49 (0.7) 


515 (2.7) 


51 (0.7) 


517 (3.0) 


2 (3.1) 


1 Lithuania 


49 (0.9) 


527 (3.7) 


51 (0.9) 


529 (3.9) 


2 (3.6) 


t England 


50 (0.9) 


539 (4.0) 


50 (0.9) 


536 (4.2) 


3 (4.2) 


Chinese Taipei 


48 (0.5) 


565 (2.6) 


52 (0.5) 


562 (2.7) 


3 (2.9) 


t Netherlands 


49 (1.1) 


533 (3.4) 


51 (1.1) 


536 (3.2) 


4 (3.2) 


Morocco 


49 (1.1) 


366 (5.6) 


51 (1.1) 


370 (4.7) 


4 (5.4) 


Norway 


50 (0.8) 


466 (2.5) 


50 (0.8) 


470 (2.8) 


4 (3.3) 


Russian Federation 


50 (0.7) 


524 (5.2) 


50 (0.7) 


528 (4.9) 


4 (3.4) 


t United States 


50 (0.5) 


517 (2.6) 


50 (0.5) 


522 (2.9) 


5 (2.5) 


Japan 


49 (0.6) 


559 (2.1) 


51 (0.6) 


564 (2.6) 


6 (3.2) 


Italy 


48 (0.8) 


496 (4.7) 


52 (0.8) 


502 (4.1) 


6 (3.7) 


Armenia 


49 (0.8) 


449 (3.4) 


51 (0.8) 


442 (3.4) 


7 (2.9) 


t Scotland 


51 (1.0) 


495 (3.5) 


49 (1 .0) 


502 (4.0) 


7 (4.3) 


Singapore 


49 (1.4) 


578 (6.2) 


51 (1.4) 


570 (6.8) 


8 (4.7) 


Iran, Islamic Rep. of 


39 (4.2) 


406 (6.0) 


61 (4.2) 


396 (4.3) 


10 (7.7) 


Moldova, Rep. of 


50 (0.8) 


501 (5.5) 


50 (0.8) 


488 (5.6) 


13 (5.2) 


Philippines 

Benchmarking Participants 


51 (1.0) 


366 (8.8) 


49 (1 .0) 


352 (6.6) 


13 (5.7) 


Indiana State, US 


52 (1.1) 


528 (3.9) 


48 (1.1) 


528 (4.1) 


0 (4.3) 


Ontario Province, Can. 


48 (1.1) 


520 (3.4) 


52 (1.1) 


525 (4.8) 


6 (4.1) 


Quebec Province, Can. 


50 (0.9) 


510 (3.0) 


50 (0.9) 


514 (3.2) 


4 (3.6) 



Gender Difference 



Girls 


Boys 


Scored Higher 


Scored Higher 







- 





I 1 1 1 1 

40 20 0 20 40 



■ Gender difference statistically significant 
□ Gender difference not statistically significant 



t Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

1 National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 



( ) Standard errors appear in parentheses. Because results are rounded to the nearest whole number, 
some totals may appear inconsistent. 
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Chapter 4 

Country by Country Profiles of 
Achievement in the Mathematics 
Cognitive Domains 

To highlight relative strengths and weaknesses within each country, 
this chapter describes in which mathematics cognitive areas each 
country is relatively strong or weak. Regardless of international 
standing, the profiles of achievement within country reveal that 
many countries performed relatively better or worse in one or more 
cognitive domains than they did overall. 

Differences in relative performance may be related to one or 
more of a number of factors, such as emphases in intended curriculum 
or widely used textbooks, differences in instruction and curriculum 
implementation, and differences in the match between instruction and 
the types of items contained in TIMSS 2003. 

Profiles of Achievement 

For each country, Exhibit 4.1 displays the difference between average 
performance in each content area and the country's average perfor- 
mance overall. The first three pages of Exhibit 4.1 show the results 
for eighth grade and the next two pages show the results for the 
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fourth grade. For each country, the average of the cognitive domain 
scores has been set to zero, so that above average or below average 
performance can be highlighted for each of the three domains. Rela- 
tively better achievement in a cognitive domain is shown when the 
circle and the lines indicating its confidence interval are completely 
above and not touching zero on the scale, and relatively worse 
achievement by a circle and its confidence interval lines completely 
below "0." 

The profiles of relative performance reveal interesting differ- 
ences among countries. Most countries show the profile of performing 
relatively better or worse in only one of the domains, or perhaps having 
a relative strength in one domain together with a relative weakness 
in another of the domains. However, a few countries were very bal- 
anced in their performance across the cognitive domains, for example, 
Belgium (Flemish) at the eighth grade and Chinese Taipei at the fourth 
grade. At the other end of the continuum, a few countries had a rela- 
tive strength or weakness in each of the three domains. For example, 
at the eighth grade, it can be seen that Bahrain performed relatively 
better in the reasoning domain and relatively worse in the knowing 
and applying domains compared to its average achievement overall. At 
the fourth grade, the only country with this pattern was Norway, with 
relatively worse performance in the knowing and applying domains, 
combined with better performance in the reasoning domain. 

Relative Strengths and Weaknesses in the Knowing Domain 

At the eighth grade, countries with relative strength in the knowing 
domain included Botswana, Bulgaria, Cyprus, Estonia, Hong Kong SAR, 
Israel, the Republic of Korea, Latvia, Lebanon, Lithuania, the Philip- 
pines, Romania, the Russian Federation, Serbia, and the Slovak Repub- 
lic. The countries that performed significantly less well in the knowing 
domain than in mathematics overall included Australia, Bahrain, Chile, 
Ghana, Iran, the Netherlands, New Zealand, Norway, Saudi Arabia, Scot- 
land, Sweden, England, and the Canadian province of Ontario. At the 
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TIMSS2003 



Exhibit 4.1: Profiles of Within-Country Relative Performance in Mathematics 

Cognitive Domains 



MATHEMATICS 
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Difference from Country's Own Average 
of Mathematics Cognitive Domain Scale Scores 
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- confidence interval (±2SE) 
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(set to 0) 



t Met guidelines for sample participation rates only after replacement schools were included (see m Korea tested the same cohort of students as other countries, but later in 2003, at the beginning of the 

Exhibit C.2). next school year. 

1 National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 

2 National Defined Population covers less than 90% of National Desired Population (see Exhibit C.1). 



TIMSS 8- PIRLS INTERNATIONAL STUDY CENTER, LYNCH SCHOOL OF EDUCATION, BOSTON COLLEGE 



55 



SOURCE: lEA's Trends in International Mathematics and Science Study (TIMSS) 2003 





























CHAPTER 4: COUNTRY BY COUNTRY PROFILES OF ACHIEVEMENT IN THE MATHEMATICS COGNITIVE DOMAINS 



TIMSS2003 



Exhibit 4.1: Profiles of Within-Country Relative Performance in Mathematics 

Cognitive Domains 



MATHEMATICS L O ) 

Grade (^Oj 



Difference from Country's Own Average 
of Mathematics Cognitive Domain Scale Scores 
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(set to 0) 



t Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

$ Nearly satisfied guidelines for sample participation rates only after replacement schools were 
included (see Exhibit C.2). 



1 National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 

2 National Defined Population covers less than 90% of National Desired Population (see Exhibit C.1). 
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TIMSS2003 



Exhibit 4.1: Profiles of Within-Country Relative Performance in Mathematics 

Cognitive Domains 



MATHEMATICS 

Grade 
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Difference from Country's Own Average 
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t Nearly satisfied guidelines for sample participation rates only after replacement schools were 
included (see Exhibit C.2). 

t Did not satisfy guidelines for sample participation rates (see Exhibit C.2). 
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Exhibit 4.1: Profiles of Within-Country Relative Performance in Mathematics 

Cognitive Domains 
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t Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 

1 National Desired Population does not cover all of International Desired Population (see Exhibit C.1). 
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TIMSS2003 



Exhibit 4.1: Profiles of Within-Country Relative Performance in Mathematics 

Cognitive Domains 
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t Met guidelines for sample participation rates only after replacement schools were included (see 
Exhibit C.2). 
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fourth grade, countries with a relative strength in the knowing domain 
were Belgium (Flemish), Italy, Singapore, the United States, and the 
US state of Indiana. Comparatively more countries at the fourth grade 
had a relative weakness in the knowing domain, including Cyprus, 
Latvia, Lithuania, the Netherlands, Norway, the Russian Federation, 
and Slovenia. 

Relative Strengths and Weaknesses in the Applying Domain 

At the eighth grade, there were fewer countries with differences 
between overall mathematics achievement and achievement in the 
applying domain than there were with such differences in the knowing 
domain. Countries with a relative strength in the applying domain at 
the eighth grade included Ghana, Singapore, and Tunisia. Those with 
a relative weakness in the applying domain included Bahrain, Mace- 
donia, and Serbia. 

At the fourth grade, Armenia, Latvia, Lithuania, and the Russian 
Federation had applying as a particular strength. Compared to per- 
formance in overall mathematics, applying was a relative weakness 
in Australia, New Zealand, Norway, the United States, the US state 
of Indiana, and the two Canadian provinces (a group including three 
English-speaking countries). 

Relative Strengths and Weaknesses in the Reasoning Domain 

Countries with the reasoning domain as a particular strength at the 
eighth grade included Bahrain, Chile, Ghana, Japan, Norway, the Pal- 
estinian National Authority, Saudi Arabia, Scotland, South Africa, and 
Sweden. Countries that performed less well in the reasoning domain 
than they did in overall mathematics included Armenia, Botswana, 
Cyprus, Hong Kong, Israel, Lebanon, Lithuania, the Philippines, 
Romania, the Russian Federation, Serbia, and Singapore. 

At the fourth grade, the participants with a relative strength in 
reasoning were Cyprus, New Zealand, Norway, Scotland, Slovenia, and 



60 



TIMSS 8- PIRLS INTERNATIONAL STUDY CENTER, LYNCH SCHOOL OF EDUCATION, BOSTON COLLEGE 



CHAPTER 4: COUNTRY BY COUNTRY PROFILES OF ACHIEVEMENT IN THE MATHEMATICS COGNITIVE DOMAINS 



the two Canadian provinces. Only two countries, Belgium (Flemish) 
and Singapore, did relatively less well in reasoning as compared to their 
overall mathematics performance. 

International Achievement Across the Cognitive Domains 

At the eighth grade across the TIMSS 2003 participants, the knowing 
domain had the most differences, with many countries showing either 
a relative strength or weakness in this area. Fifteen countries performed 
better in the knowing domain than they did in mathematics overall, 
and 12 countries and the Canadian province of Ontario performed 
worse. The applying domain was the cognitive area least likely to 
feature either relatively strong or relatively weak performance. Only 
three countries performed better in the applying domain than they did 
in mathematics overall (Ghana, Singapore, and Tunisia) and only three 
countries performed worse (Bahrain, Macedonia, and Serbia). 

In the reasoning domain at the eighth grade, 10 countries per- 
formed relatively better than they did in mathematics overall and 12 
countries did less well. The countries making up each of the two groups 
included those from very different parts of the world geographically 
and with disparate cultures and mathematics traditions. For example, 
the countries with a relative strength in the reasoning domain were 
Bahrain, Chile, Ghana, Japan, Norway, the Palestinian National 
Authority, Saudi Arabia, Scotland, South Africa, and Sweden. 

At the fourth grade, looking across the participating coun- 
tries, about the same number of differences (strengths or weaknesses) 
occurred in each of the cognitive domains. However, several more 
countries showed a relative weakness in the knowing cognitive domain 
(seven) than had this domain as a relative strength (five). Similarly, 
more countries had a relative weakness in the applying domain (seven) 
than had this domain as a relative strength (four). In comparison, more 
countries showed a relative strength in the reasoning domain (seven) 
than showed this domain as a relative weakness (two). 
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Appendix A 

Mathematics Cognitive Domains 
Framework: TIMSS 2003 
Developmental Project Fourth and 
Eighth Grades 

To respond correctly to TIMSS test items, students need to be famil- 
iar with the mathematics content being assessed, but they also need 
to draw on a range of cognitive skills. The first domain, knowing facts, 
procedures, and concepts, covers what the student needs to know, while 
the second, applying knowledge and conceptual understanding, focuses on 
the ability of the student to apply what he or she knows to solve prob- 
lems or answer questions. The third domain, reasoning, goes beyond 
the solution of routine problems to encompass unfamiliar situations, 
complex contexts, and multi-step problems. 

Knowing Facts, Procedures, and Concepts 

Facility in using mathematics, or reasoning about mathematical situations, 
depends on mathematical knowledge and familiarity with mathematical 
concepts. The more relevant knowledge a student is able to recall and the 
wider the range of concepts he or she has understood, the greater the 
potential for engaging a wide range of problem-solving situations and for 
developing mathematical understanding. 
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Without access to a knowledge base that enables easy recall of the 
language and basic facts and conventions of number, symbolic representa- 
tion, and spatial relations, students would bnd purposeful mathematical 
thinking impossible. Facts encompass the factual knowledge that provides 
the basic language of mathematics, and the essential mathematical facts 
and properties that form the foundation for mathematical thought. 

Procedures form a bridge between more basic knowledge and 
the use of mathematics for solving routine problems, especially those 
encountered by many people in their daily lives. In essence a fluent 
use of procedures entails recall of sets of actions and how to carry them 
out. Students need to be efficient and accurate in using a variety of 
computational procedures and tools. They need to see that particular 
procedures can be used to solve entire classes of problems, not just 
individual problems. 

Knowledge of concepts enables students to make connections 
between elements of knowledge that, at best, would otherwise be 
retained as isolated facts. It allows them to make extensions beyond 
their existing knowledge, judge the validity of mathematical statements 
and methods, and create mathematical representations. 

This cognitive domain covers the following behaviors: 



Recall 


Recall definitions; terminology; number properties; 
geometric properties; and notation (e.g., a x b = ab, 
a + a + a = 3a). 


Recognize 


Recognize mathematical objects, shapes, numbers 
and expressions. Recognize mathematical entities 
that are mathematically equivalent, e.g. areas of 
parts of figures to represent fractions, equivalent 
familiar fractions, decimals and percents; simple 
algebraic expressions that represent a straightfor- 
ward situation (eighth grade); different orientations 
of simple geometric figures; and the nets of simple 
geometric figures (eighth grade). 
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Compute 


Carry out algorithmic procedures for +, -, x, -r, 
or a combination of these with whole numbers, 
fractions, decimals and integers. Approximate 
numbers to estimate computations. Carry out 
routine algebraic procedures. 


Retrieve 


Retrieve information from graphs, tables or other 
sources; read simple scales. 


Measure 


Use measuring instruments to draw lines, angles, 
and shapes to given specifications; use units of mea- 
surement appropriately; and estimate measures. 


Know 


Know concepts (e.g., place value; rounding; that 
length, area and volume are conserved under 
certain conditions; equal and unequal chance). 



Classify/Order Classify/group objects, shapes, numbers and 



expressions according to common properties; 
make correct decisions about class membership; 
and order numbers and objects by attributes. 

Applying Knowledge and Understanding 

Problem solving is a central aim, and often means, of teaching school 
mathematics, and hence this and supporting skills (e.g., select, repre- 
sent, model) feature prominently in the applying knowledge and concep- 
tual understanding domain. In items aligned with this domain, students 
need to apply mathematical knowledge of facts, skills, and procedures 
or understanding of mathematical concepts to create representations 
and solve problems. Representation of ideas forms the core of math- 
ematical thinking and communication, and the ability to create equiva- 
lent representations are fundamental to success in the subject. 

The problem settings are more routine than those aligned with 
the reasoning domain. The routine problems will typically have been 
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standard in classroom exercises designed to provide practice in particu- 
lar methods or techniques. Some of these problems will have been in 
words that set the problem situation in a quasi-real context. Though 
they range in difficulty, each of these types of "textbook" problems is 
expected to be sufficiently familiar to students that they will essentially 
involve selecting and applying learned procedures. 

Problems may be set in real-life situations, or may be con- 
cerned with purely mathematical questions involving, for example, 
numeric or algebraic expressions, functions, equations, geometric 
figures, or statistical data sets. Therefore, problem solving is included 
not only in the applying knowledge and conceptual understanding domain, 
with emphasis on the more familiar and routine tasks, but also in the 
reasoning domain. 

This cognitive domain covers the following behaviors: 



Select 


Select an efficient /appropriate operation, method 
or strategy for solving problems where there is a 
known algorithm or method of solution. Select 
simple algebraic expressions which represent 
straightforward situations (fourth grade). Select 
the nets of simple geometric figures (fourth grade). 
Select appropriate algorithms or formulas. 


Represent 


Display mathematical information and data in 
diagrams, tables, charts, or graphs, and gener- 
ate equivalent representations for a given math- 
ematical entity or relationship. 


Model 


Generate an appropriate model, such as an equa- 
tion or diagram for solving a routine problem. 


Implement 


Follow and execute a set of mathematical 
instructions. 
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Solve Routine Solve routine problems (i.e., problems similar to 
Problems those target students are likely to have encoun- 

tered in class). For example, use geometric prop- 
erties to solve problems; compare and match 
different representations of data (eighth grade) 
and use data from charts, tables, graphs, and 
maps to solve routine problems. 

Reasoning 

Reasoning mathematically involves the capacity for logical, systematic 
thinking. It includes intuitive and inductive reasoning based on pat- 
terns and regularities that can be used to arrive at solutions to non- 
routine problems. Non-routine problems are problems that are very 
likely to be unfamiliar to students. They make cognitive demands over 
and above those needed for solution of routine problems, even when 
the knowledge and skills required for their solution have been learned. 
Non-routine problems may be purely mathematical or may have real- 
life settings. Both types of items involve transfer of knowledge and 
skills to new situations, and interactions among reasoning skills are 
usually a feature. Problems requiring reasoning may do so in different 
ways, because of the novelty of the context or the complexity of the 
situation or because any solution to the problem must involve several 
steps, perhaps drawing on knowledge and understanding from differ- 
ent areas of mathematics. 

Even though of the other behaviors listed within the rea- 
soning domain are those that may be drawn on in thinking about 
and solving novel or complex problems, each by itself represents a 
valuable outcome of mathematics education, with the potential to 
influence learners' thinking more generally. For example, reasoning 
involves the ability to observe and make conjectures. It also involves 
making logical deductions based on specific assumptions and rules, 
and justifying results. 
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This cognitive domain covers the following behaviors 



Analyze 


Determine and describe or use relationships 
between variables or objects in mathemati- 
cal situations; use proportional reasoning 
(fourth grade); decompose geometric figures 
to simplify solving a problem; draw the net of 
a given unfamiliar solid; visualize transforma- 
tions of three-dimensional figures; compare 
and match different representations of the 
same data (fourth grade); and make valid 
inferences from given information. 


Generalize 


Extend the domain to which the result of 
mathematical thinking and problem solving is 
applicable by restating results in more general 
and more widely applicable terms. 


Synthesize/ 

Integrate 


Combine (various) mathematical procedures 
to establish results, and combine results to 
produce a further result. Make connections 
between different elements of knowledge and 
related representations, and make linkages 
between related mathematical ideas. 


Justify 


Provide a justification for the truth or falsity 
of a statement by reference to mathematical 
results or properties. 


Solve Non-Routine 
Problems 


Solve problems set in mathematical or 
real life contexts where target students are 
unlikely to have encountered closely similar 
items, and apply mathematical procedures in 
unfamiliar or complex contexts. Use geomet- 
ric properties to solve non-routine problems. 
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Appendix B 

Overview of Procedures 

TIMSS 2003 Developmental Project 

Process for Establishing the Mathematics Cognitive Domains for 
Scaling and Reporting 

As explained in Chapter 1, developing reliable and valid achievement 
scales in the cognitive domains began with conducting a meeting of 
mathematics experts to examine the classification of the TIMSS 2003 
items. Hosted by the IEA Secretariat in Amsterdam, 10 participants (see 
below) met in February 2005. 

Participants in Mathematics Expert Meeting 

Amsterdam, February 2005 

Khattab Mohammad Abu Lebdeh - Jordan 

Yu-Hsien Chang - Chinese Taipei 

Tandi Clausen-May - England 

Robert Garden - New Zealand 

Barbara Japelj - Slovenia 

Michael Martin - TIMSS Study Director 

Ina Mullis - TIMSS Study Director 

Peter Nystrom - Sweden 

David Robitaille - Canada 

Graham Ruddock - England 
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Based on an iterative process of discussion and classification of 
items, the meeting participants worked with the four cognitive domains 
specified in the TIMSS 2003 Framework - knowing facts and procedures; 
using concepts; solving routine problems; and reasoning - to devise the 
three cognitive domains used as the basis for this report. Essentially, 
the "knowing facts and procedures" and the "using concepts" domains 
in the TIMSS 2003 Framework were combined, and then distinctions 
between the combined domain and solving routine problems were 
clarified. Finally, distinctions were clarified between these two domains 
and reasoning. This process led to the three domains - knowing facts, 
procedures, and concepts; applying knowledge and understanding; and 
reasoning (see Appendix A). (For the TIMSS 2007 Framework, the par- 
ticipating countries suggested that these be shortened to knowing, apply- 
ing, and reasoning for both mathematics and science.) 

Subsequent to the Amsterdam meeting, the cognitive domains 
devised for the developmental project were reviewed by the TIMSS 2007 
Science and Mathematics Item Review Committee (SMIRC). Hosted 
by the National Foundation for Educational Research in England and 
Wales (the institution of the IEA Chair and the TIMSS 2007 Mathemat- 
ics Coordinator), this meeting was held in April 2005 in London. In 
particular, the SMIRC mathematics experts endorsed reporting accord- 
ing to the three cognitive domains and worked to further refine and 
clarify the description of each domain (see below for participants). 
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Mathematics Participants in TIMSS 2007 Science and Mathematics Item 
Review Committee Meeting 
London, April 2005 

Khattab Mohammad Abu Lebdeh - Jordan 
Alka Arora - TIMSS Research Associate 
Kiril Bankov - Bulgaria 
Robert Garden - New Zealand 
Liv Sissel Gronmo - Norway 
Chen-yung Lin - Chinese Taipei 
Mary Lindquist - United States 
Ina Mullis - TIMSS Study Director 

Graham Ruddock - TIMSS 2007 Mathematics Coordinator 
Hanako Senuma - Japan 

Characteristics of Items Within Cognitive Domains 

IEA's TIMSS Er PIRLS International Study Center (ISC) examined the 
spread of the items within the three domains according to item type 
(constructed-response or multiple -choice), content domain (algebra, 
geometry, etc.), and average difficulty (mean percent correct) to 
ensure there was sufficient coverage within each domain. As shown 
in Exhibit B.l, the classification resulted in a substantial number of 
items in each cognitive domain at both eighth grade (first page) and 
fourth grade (second page). Of the 194 items at the eighth grade, 65 
were classified in the knowing cognitive domain, 93 in the apply- 
ing cognitive domain, and 36 in the reasoning cognitive domain. Of 
the 159 items at the fourth grade, 58 were classified in the knowing 
cognitive domain, 63 in the applying cognitive domain, and 38 in the 
reasoning cognitive domain. 

Within each cognitive domain, there was a very good spread of 
items in terms of item type (constructed-response or multiple -choice) 
at both eighth and fourth grades. Equivalent percentages of apply- 
ing items were multiple-choice and constructed-response. As would 
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TIMSS2003 



Exhibit B.1: Characteristics of Items Within Cognitive Domains mathematics l O J 

Grade 



Number of Items by Item Type and Cognitive Domains 



Item Type 


Cognitive Domains 


Total 


Knowing 


Applying 


Reasoning 


Constructed Response 


11 


34 


21 


66 


Multiple Choice 


54 


59 


15 


128 


Total 


65 


93 


36 


194 



Percent of Score Points by Item Type and Cognitive Domains 



Item Type 


Cognitive Domains 


Total 


Knowing 


Applying 


Reasoning 


Score Points 


Constructed Response 


14% 


47% 


39% 


85 


Multiple Choice 


42% 


46% 


12% 


128 


Total 


31% 


46% 


23% 


213 



Number of Items by Content Domain and Cognitive Domain 



Content Domain 


Cognitive Domains 


Total 


Knowing 


Applying 


Reasoning 


Number 


21 


31 


5 


57 


Algebra 


22 


12 


13 


47 


Measurement 


7 


22 


2 


31 


Geometry 


10 


12 


9 


31 


Data 


5 


16 


7 


28 


Total 


65 


93 


36 


194 



Percent of Score Points by Content Domain and Cognitive Domain 



Content Domain 


Cognitive Domains 


Total 


Knowing 


Applying 


Reasoning 


Score Points 


Number 


35% 


55% 


10% 


60 


Algebra 


43% 


23% 


34% 


53 


Measurement 


21% 


73% 


6% 


33 


Geometry 


30% 


36% 


33% 


33 


Data 


15% 


53% 


32% 


34 


Total 


31% 


46% 


23% 


213 



Mean Percent Correct by Content Domain and Cognitive Domain 



Item Difficulties 


Cognitive Domains 


Total 


(Mean Percent Correct) 


Knowing 


Applying 


Reasoning 


Number 


50% 


43% 


36% 


45% 


Algebra 


49% 


45% 


29% 


42% 


Measurement 


55% 


37% 


41% 


41% 


Geometry 


51% 


50% 


36% 


46% 


Data 


53% 


46% 


34% 


44% 


Total 


50% 


43% 


33% 


44% 
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TIMSS2003 



Exhibit B.1: Characteristics of Items Within Cognitive Domains 



MATHEMATICS 

Grade 



Number of Items by Item Type and Cognitive Domains 



Item Type 


Cognitive Domains 


Total 


Knowing 


Applying 


Reasoning 


Constructed Response 


17 


28 


24 


69 


Multiple Choice 


41 


35 


14 


90 


Total 


58 


63 


38 


159 1 



Percent of Score Points by Item Type and Cognitive Domains 



Item Type 


Cognitive Domains 


Total 


Knowing 


Applying 


Reasoning 


Score Points 


Constructed Response 


24% 


38% 


38% 


76 


Multiple Choice 


46% 


39% 


16% 


90 


Total 


36% 


39% 


26% 


166 



Number of Items by Content Domain and Cognitive Domain 



Content Domain 


Cognitive Domains 


Total 


Knowing 


Applying 


Reasoning 


Number 


25 


19 


19 


63 


Patterns and Relationships 


2 


13 


8 


23 


Measurement 


10 


18 


4 


32 


Geometry 


17 


6 


1 


24 


Data 


4 


7 


6 


17 


Total 


58 


63 


38 


159 



Percent of Score Points by Content Domain and Cognitive Domain 



Content Domain 


Cognitive Domains 


Total 


Knowing 


Applying 


Reasoning 


Score Points 


Number 


37% 


29% 


34% 


68 


Patterns and Relationships 


8% 


54% 


38% 


24 


Measurement 


31% 


56% 


13% 


32 


Geometry 


72% 


24% 


4% 


25 


Data 


24% 


41% 


35% 


17 


Total 


36% 


39% 


26% 


166 



Mean Percent Correct by Content Domain and Cognitive Domain 



Item Difficulties 


Cognitive Domains 


Total 


(Mean Percent Correct) 


Knowing 


Applying 


Reasoning 


Number 


63% 


56% 


37% 


53 


Patterns and Relationships 


63% 


57% 


36% 


50 


Measurement 


65% 


47% 


39% 


52 


Geometry 


60% 


53% 


43% 


57 


Data 


69% 


56% 


58% 


60 


Total 


53% 


53% 


40% 


54 



1 There were 1 61 items on the fourth grade mathematics assessment. Following item review, two items 
were deleted, and were not included in the cognitive domain scaling. 
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be expected, however, at both grades a relatively higher percentage 
of items in the knowing domain were multiple-choice, and a com- 
mensurately higher percentage of items in the reasoning domain were 
constructed-response. Often, the multiple -choice format is a cost-effec- 
tive way to assess specific knowledge, while the constructed-response 
format may be required in complex problem-solving situations involv- 
ing multiple strategies. 

Despite some unevenness, there was good spread across content 
domains within each of the three cognitive domains. At eighth grade, 
it would have been preferable to have a higher proportion of number 
items in the reasoning domain (an effort is being made to address this 
in TIMSS 2007). That the distribution for measurement is concentrated 
in the applying domain makes some sense, since by eighth grade stu- 
dents should know about basic measurement tools and units. (In the 
TIMSS 2007 Framework, aspects of measurement were incorporated 
into the number and geometry content domains because there is little 
emphasis on measurement in eighth-grade mathematics curricula 
around the world). 

Because algebra is generally not taught as a formal subject in 
primary school, only introductory concepts about patterns and relation- 
ships are assessed at the fourth grade. As such, a higher proportion of 
patterns and relationship items in the knowing category would have 
been preferable at the fourth grade. (In the TIMSS 2007 Framework, 
the patterns and relationships content domain has been incorporated 
into the number content domain.) Also, a higher proportion of mea- 
surement items in the reasoning domain would have been better. The 
low coverage of geometry in the reasoning domain is understandable, 
since this is a subject little emphasized at the fourth grade. (In the 
TIMSS 2007 Framework, the geometry content domain, now called 
geometric shapes and measures, has been recast to better describe the 
fourth-grade curricula of participating countries.) 

Finally, Exhibit B.l also shows a good range in item difficulty 
(mean percentage correct) internationally, on average, within each of 
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the three cognitive domains. As would be anticipated, at both grades 
there was the same overall pattern, with the reasoning items the most 
difficult. Essentially, this pattern of the items in the reasoning domain 
being more difficult than the knowing or applying items was consistent 
across the content domains at both grades. 

Constructing Achievement Scales in the Mathematics 
Cognitive Domains 

The scaling methodology was identical to that used to report mathemat- 
ics achievement results and achievement in the mathematics content 
domains in the TIMSS 2003 International Report. It is described in 
detail in Gonzalez, Galia, and Li (2004). 

The TIMSS 2003 goals of broad coverage of the mathemat- 
ics and science curriculum and of measuring trends across assess- 
ments necessitated a complex matrix-sampling booklet design, with 
individual students responding to a subset of the mathematics and 
science items in the assessment but not the entire assessment item 
pool. Given the complexities of the data collection and the need to 
have student scores on the entire assessment for analysis and report- 
ing purposes, TIMSS 2003 relied on Item Response Theory (IRT) 
scaling to describe student achievement on the assessment and to 
provide accurate measures of trends from previous assessments. The 
TIMSS IRT scaling approach used multiple imputation, or "plausible 
values" methodology, to obtain proficiency scores in mathematics 
and science for all students, even though each student responded to 
only a part of the assessment item pool. To enhance the reliability of 
the student scores, the TIMSS scaling combined student responses to 
the items they were administered with information about students' 
backgrounds, a process known as "conditioning." 

Using routine TIMSS procedures, three distinct IRT scaling 
models, depending on item type and scoring procedure, were used 
in constructing achievement scales for the mathematics cognitive 
domains. Each scaling model is a "latent variable" model that describes 
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the probability that a student will respond in a specibc way to an item 
in terms of the respondent's proficiency, which is an unobserved or 
"latent" trait, and various characteristics (or "parameters") of the item. 
A three -parameter model was used with multiple -choice items, which 
were scored as correct or incorrect, and a two-parameter model for 
constructed-response items with just two response options, which also 
were scored as correct or incorrect. Since each of these item types has 
just two response categories, they are known as dichotomous items. A 
partial credit model was used with polytomous constructed-response 
items, i.e., those with more than two score points. 

Item Calibration 

The first step in constructing the cognitive domain scales was to esti- 
mate the IRT model parameters for each item on each of the cogni- 
tive domain scales. This procedure, known as item calibration, was 
implemented using the PARSCALE software applied to a self-weight- 
ing random sample of 1000 students from each country's TIMSS 2003 
student sample. Using student samples of equal size ensured that the 
data from each country contributed equally to the item calibration, 
while keeping the amount of data to be analyzed to a reasonable size. 

At the fourth and eighth grades, separate calibrations were con- 
ducted for each of the three mathematics cognitive domains: knowing, 
applying, and reasoning (abbreviated labels). At the eighth grade, the 
calibrations were based on 46,000 student records; 1,000 from each of 
the 46 countries that participated in the 2003 assessment. At the fourth 
grade, the calibrations were based on 26,000 student records, 1,000 
from each of the 26 countries that participated in the 2003 assessment 
at the fourth grade. 
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Evaluating the Fit of the IRT Models 

After the calibrations were completed, checks were performed to verify 
that the item parameters obtained from PARSCALE for the three cog- 
nitive scales were a good fit for the data. An item is said to fit the IRT 
model when the empirical distribution of student responses (i.e., the 
proportion of correct student responses at various levels of student 
proficiency) closely matches the theoretical item response curve con- 
structed from the estimated item parameters. For every item at both 
grades, the empirical and theoretical distributions were plotted and 
compared. 

Generating IRT Proficiency Scores 

Following item calibration, Educational Testing Service's MGROUP 
program was used to generate the IRT proficiency scores for the cogni- 
tive domain scales. This program takes as input the students' responses 
to the items they were given, the item parameters estimated at the 
calibration stage, and the conditioning variables derived from student 
background variables, and generates as output the plausible values that 
represent student proficiency. 

Plausible values generated by the conditioning program are 
initially on the same scale as the item parameters used to estimate 
them. This scale metric is generally not useful for reporting purposes 
since it is somewhat arbitrary, ranges between approximately -3 and 
+3, and has a mean of zero across all countries. The plausible values 
for each cognitive domain scale were transformed to the same metric 
as the overall mathematics scale in 2003, as was done for the content 
domain scaling in 2003. Thus, for the eighth grade, each of the three 
cognitive domain scales were set to have a mean of 467 and standard 
deviation of 100, and for the fourth grade, a mean of 495 and standard 
deviation of 100. 
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Reliability 

Exhibit B.2 displays the reliability coefficient for each country for the 
mathematics test overall and for the knowing, applying, and reasoning 
cognitive domains. The first page shows the reliabilities for the eighth 
grade and the second page shows the reliabilities for the fourth grade. 
Reliability was measured as the ratio of sampling variance to sampling 
variance plus imputation variance. This approach is more suitable for 
multiple -matrix -sampling designs where students respond to relatively 
few items than classical reliability methods (such as the well-known 
Kuder-Richardson formulas) that are affected by the number of items 
taken by the student. Reliability coefficients greater than .80 are gener- 
ally considered acceptable for such designs. 

At both grade levels, despite some variation, reliabilities gen- 
erally were high for most countries. The international median (the 
median of the reliability coefficients for all countries) was .96 at the 
eighth grade and .97 at the fourth grade for the overall mathematics 
assessment. At the eighth grade, the median reliabilities for the cog- 
nitive domains were .93 for knowing, .96 for applying, and .88 for 
reasoning. At the fourth grade, they were .92 for knowing, .93 for 
applying, and .91 for reasoning. 
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TIMSS2003 



Exhibit B.2: Reliabilities of Overall Mathematics and Cognitive Domains 



MATHEMATICS 

Grade 



Countries 


Reliabilities of Overall Mathematics 
and Cognitive Domains 




Overall 


Knowing 


Applying 


Reasoning 


Armenia 


0.97 


0.96 


0.92 


0.93 



Australia 


0.99 


0.97 


0.99 


0.97 


Bahrain 


0.83 


0.51 


0.72 


0.46 


Belgium (Flemish) 


0.95 


0.98 


0.97 


0.89 


Botswana 


0.72 


0.67 


0.73 


0.56 


Bulgaria 


0.96 


0.99 


0.95 


0.88 


Chile 


0.88 


0.83 


0.96 


0.67 


Chinese Taipei 


0.96 


0.99 


0.96 


0.92 


Cyprus 


0.79 


0.33 


0.91 


0.90 


Egypt 


0.95 


0.96 


0.98 


0.85 


England 


0.99 


0.93 


0.99 


0.88 


Estonia 


0.96 


0.95 


0.96 


0.87 


Ghana 


0.87 


0.80 


0.92 


0.80 


Hong Kong, SAR 


0.95 


0.88 


0.97 


1.00 


Hungary 


0.99 


0.96 


0.95 


0.91 


Indonesia 


0.98 


0.98 


0.98 


0.93 


Iran, Islamic Rep. of 


0.97 


0.79 


0.86 


0.48 


Israel 


0.96 


0.92 


0.97 


0.96 


Italy 


0.97 


0.97 


0.97 


0.95 


Japan 


0.92 


0.92 


0.96 


0.91 


Jordan 


0.99 


0.93 


0.97 


0.72 


Korea, Rep. of 


0.71 


0.82 


0.67 


0.79 


Latvia 


0.97 


0.97 


0.96 


0.88 


Lebanon 


0.97 


0.95 


0.96 


0.86 


Lithuania 


0.99 


0.78 


0.96 


0.97 


Macedonia, Rep. of 


0.98 


0.78 


0.92 


0.86 


Malaysia 


0.98 


0.98 


0.95 


0.91 


Moldova, Rep. of 


0.98 


0.88 


0.96 


0.93 


Morocco 


0.77 


0.81 


0.56 


0.50 


Netherlands 


0.98 


0.98 


1.00 


0.87 


New Zealand 


0.97 


0.97 


0.98 


0.96 


Norway 


0.94 


0.95 


0.82 


0.84 


Palestinian Nat'l Auth. 


0.96 


0.90 


0.85 


0.95 


Philippines 


0.98 


0.95 


0.98 


0.87 


Romania 


0.99 


0.96 


0.99 


0.92 


Russian Federation 


0.91 


0.93 


0.96 


0.82 


Saudi Arabia 


0.95 


0.84 


0.98 


0.94 


Scotland 


0.93 


0.92 


0.97 


0.99 


Serbia 


0.91 


0.83 


0.86 


0.74 


Singapore 


0.96 


0.98 


0.97 


0.97 


Slovak Republic 


0.96 


0.97 


0.92 


0.92 


Slovenia 


0.91 


0.68 


0.78 


0.74 


South Africa 


0.94 


0.94 


0.91 


0.97 


Sweden 


0.95 


0.93 


0.91 


0.84 


Tunisia 


0.94 


0.72 


0.83 


0.52 


United States 


0.98 


0.99 


0.99 


0.87 



International Median 0.96 0.93 0.96 0.88 



Benchmark Participants: 



Basque Country, Spain 


0.86 


0.76 


0.93 


0.89 


Ontario Province, Can. 


0.95 


0.89 


0.97 


0.97 


Quebec Province, Can. 


0.98 


0.96 


0.98 


0.87 


Indiana State, US 


1.00 


0.97 


0.97 


0.84 
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TIMSS2003 



Exhibit B.2: Reliabilities of Overall Mathematics and Cognitive Domains 



MATHEMATICS 

Grade 




Countries 


Reliabilities of Overall Mathematics 
and Cognitive Domains 




Overall 


Knowing 


Applying 


Reasoning 


Armenia 


0.99 


0.94 


0.99 


0.97 


Australia 


0.97 


0.98 


0.98 


0.91 


Belgium (Flemish) 


0.99 


0.83 


0.76 


0.91 


Chinese Taipei 


0.94 


0.88 


0.89 


0.79 


Cyprus 


0.98 


0.89 


0.91 


0.83 


England 


0.98 


0.84 


0.95 


0.97 


Hong Kong, SAR 


0.95 


0.95 


0.93 


0.91 


Hungary 


0.95 


0.91 


0.96 


0.93 


Iran, Islamic Rep. of 


0.93 


0.83 


0.95 


0.95 


Italy 


0.99 


0.98 


0.93 


0.96 


Japan 


0.90 


0.71 


0.74 


0.77 


Latvia 


0.97 


0.97 


0.87 


0.90 


Lithuania 


0.99 


0.99 


0.88 


0.94 


Moldova, Rep. of 


0.98 


0.92 


0.99 


0.94 


Morocco 


0.95 


0.93 


0.98 


0.91 


Netherlands 


0.91 


0.92 


0.79 


0.59 


New Zealand 


0.99 


0.88 


0.87 


0.80 


Norway 


0.97 


0.89 


0.92 


0.87 


Philippines 


0.99 


0.97 


0.98 


0.93 


Russian Federation 


0.99 


0.95 


1.00 


0.99 


Scotland 


0.95 


0.92 


0.91 


0.87 


Singapore 


0.99 


0.95 


1.00 


1.00 


Slovenia 


0.97 


0.80 


0.86 


0.98 


Tunisia 


0.93 


0.88 


0.93 


0.91 


United States 


0.99 


0.96 


0.85 


0.93 


International Median 


0.97 


0.92 


0.93 


i 091 


Benchmark Participants: 


Ontario Province, Can. 


0.98 


0.95 


0.99 


0.98 


Quebec Province, Can. 


0.94 


0.84 


0.85 


0.91 


Indiana State, US 


1.00 


0.79 


0.93 


0.75 
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Correlations 

Exhibit B.3 presents the Pearson correlation coefficient indicating the 
linear relationship between achievement in each cognitive domain and 
achievement on the overall mathematics assessment for each of the 
TIMSS 2003 countries. The first page shows the correlations for the 
eighth grade and the second page the correlations for the fourth grade. 
All of the correlations are substantial, indicating that high performance 
in each of the three cognitive domains is likely to be associated with 
high performance on the mathematics assessment overall. This means 
proficiency in each of the domains is an important contributor to math- 
ematics proficiency in general. 

At eighth grade, correlations were highest for knowing and 
applying, with a median correlation with overall mathematics achieve- 
ment of .88 in each case. This means that students with high scores in 
these domains were equally likely to have high scores on mathematics 
overall. The correlation between reasoning and overall achievement 
was generally lower, with a median correlation of .77 (consistent with 
the somewhat lower reliability of the reasoning scale). This means that 
students with high scores in the reasoning domain also were likely to 
have high scores on mathematics overall, but somewhat less likely than 
students with high scores in the knowing or applying domains. 

At the fourth grade, correlations between achievement in 
the cognitive domains and overall mathematics were more uniform, 
with correlations of .84 for the knowing domain, .86 for the apply- 
ing domain, and .83 for the reasoning domain. This means that stu- 
dents with high scores in any one of the three cognitive domains were 
equally likely to have high scores on mathematics overall. 

Correlations between the three cognitive scales are presented in 
Exhibit B.4 for the eighth grade (first page) and for the fourth grade 
(second page). As would be expected of cognitive domains within 
a single subject area, mathematics, country-level correlations at the 
eighth grade were generally moderate to high, with international 
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TIMSS2003 



Exhibit B.3: Correlations of Mathematics Cognitive Domains with Overall Mathematics 



MATHEMATICS 

Grade 




Countries 


Pearson Correlations of 
Mathematics Cognitive Domains 
with Overall Mathematics 




Knowing 


Applying 


Reasoning 


Armenia 


0.85 


0.87 


0.79 


Australia 


0.89 


0.90 


0.81 


Bahrain 


0.81 


0.81 


0.69 


Belgium (Flemish) 


0.91 


0.91 


0.83 


Botswana 


0.75 


0.76 


0.64 


Bulgaria 


0.88 


0.88 


0.76 


Chile 


0.85 


0.85 


0.71 


Chinese Taipei 


0.92 


0.93 


0.86 


Cyprus 


0.86 


0.87 


0.76 


Egypt 


0.84 


0.84 


0.73 


England 


0.89 


0.90 


0.79 


Estonia 


0.88 


0.89 


0.79 


Ghana 


0.65 


0.68 


0.54 


Hong Kong, SAR 


0.89 


0.90 


0.82 


Hungary 


0.90 


0.91 


0.82 


Indonesia 


0.86 


0.86 


0.71 


Iran, Islamic Rep. of 


0.81 


0.81 


0.70 


Israel 


0.89 


0.89 


0.79 


Italy 


0.88 


0.88 


0.77 


Japan 


0.90 


0.90 


0.83 


Jordan 


0.86 


0.86 


0.76 


Korea, Rep. of 


0.90 


0.91 


0.83 


Latvia 


0.87 


0.88 


0.79 


Lebanon 


0.81 


0.82 


0.63 


Lithuania 


0.88 


0.89 


0.78 


Macedonia, Rep. of 


0.86 


0.87 


0.75 


Malaysia 


0.89 


0.90 


0.78 


Moldova, Rep. of 


0.85 


0.85 


0.74 


Morocco 


0.70 


0.70 


0.56 


Netherlands 


0.90 


0.91 


0.81 


New Zealand 


0.88 


0.89 


0.77 


Norway 


0.85 


0.86 


0.77 


Palestinian Nat 1 1 Auth. 


0.82 


0.83 


0.72 


Philippines 


0.82 


0.83 


0.71 


Romania 


0.89 


0.90 


0.78 


Russian Federation 


0.88 


0.89 


0.76 


Saudi Arabia 


0.70 


0.71 


0.58 


Scotland 


0.89 


0.90 


0.79 


Serbia 


0.89 


0.89 


0.80 


Singapore 


0.92 


0.92 


0.85 


Slovak Republic 


0.89 


0.90 


0.82 


Slovenia 


0.87 


0.88 


0.75 


South Africa 


0.82 


0.83 


0.70 


Sweden 


0.87 


0.87 


0.75 


Tunisia 


0.74 


0.75 


0.57 


United States 


0.91 


0.91 


0.82 


International Median 


0.88 


0.88 


0.77 


Benchmark Participants: 


Basque Country, Spain 


0.83 


0.84 


0.71 


Ontario Province, Can. 


0.86 


0.88 


0.74 


Quebec Province, Can. 


0.84 


0.87 


0.74 


Indiana State, US 


0.81 


0.82 


0.73 
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TIMSS2003 



Exhibit B.3: 



Correlations of Mathematics Cognitive Domains with Overall Mathematics 



MATHEMATICS 

Grade 




Countries 


Pearson Correlations of 
Mathematics Cognitive Domains 
with Overall Mathematics 




Knowing 


Applying 


Reasoning 


Armenia 


0.81 


0.84 


0.77 


Australia 


0.86 


0.87 


0.84 


Belgium (Flemish) 


0.80 


0.83 


0.78 


Chinese Taipei 


0.82 


0.84 


0.81 


Cyprus 


0.85 


0.88 


0.84 


England 


0.87 


0.89 


0.85 


Hong Kong, SAR 


0.81 


0.84 


0.81 


Hungary 


0.85 


0.88 


0.83 


Iran, Islamic Rep. of 


0.78 


0.80 


0.71 


Italy 


0.86 


0.88 


0.83 


Japan 


0.83 


0.86 


0.82 


Latvia 


0.84 


0.87 


0.83 


Lithuania 


0.85 


0.87 


0.83 


Moldova, Rep. of 


0.85 


0.88 


0.83 


Morocco 


0.72 


0.74 


0.63 


Netherlands 


0.77 


0.82 


0.76 


New Zealand 


0.87 


0.88 


0.86 


Norway 


0.82 


0.85 


0.79 


Philippines 


0.82 


0.83 


0.77 


Russian Federation 


0.85 


0.88 


0.85 


Scotland 


0.84 


0.86 


0.81 


Singapore 


0.85 


0.89 


0.87 


Slovenia 


0.84 


0.86 


0.83 


Tunisia 


0.75 


0.77 


0.66 


United States 


0.85 


0.88 


0.85 


International Median 


0.84 


0.86 


1 0-83 i 


Benchmark Participants: 


Ontario Province, Can. 


0.84 


0.86 


0.83 


Quebec Province, Can. 


0.82 


0.84 


0.80 


Indiana State, US 


0.77 


0.79 


0.75 



TIMSS Er PIRLS INTERNATIONAL STUDY CENTER, LYNCH SCHOOL OF EDUCATION, BOSTON COLLEGE 



87 



SOURCE: lEA's Trends in International Mathematics and Science Study (TIMSS) 2003 














APPENDIX B: OVERVIEW OF PROCEDURES TIMSS 2003 DEVELOPMENTAL PROJECT 



TIMSS2003 



Exhibit B.4: Correlations of Mathematics Cognitive Domains 



MATHEMATICS 

Grade 



o 

O 



Pearson Correlations for 



Mathematics Cognitive Domains 



Countries 








Knowing 

Reasoning 


Applying 

Reasoning 


Armenia 


0.91 


0.79 


0.86 


Australia 


0.95 


0.83 


0.84 


Bahrain 


0.91 


0.75 


0.78 


Belgium (Flemish) 


0.95 


0.85 


0.85 


Botswana 


0.89 


0.74 


0.76 


Bulgaria 


0.95 


0.80 


0.80 


Chile 


0.94 


0.78 


0.76 


Chinese Taipei 


0.97 


0.89 


0.88 


Cyprus 


0.95 


0.82 


0.80 


Egypt 


0.95 


0.81 


0.82 


England 


0.95 


0.80 


0.82 


Estonia 


0.94 


0.82 


0.82 


Ghana 


0.74 


0.60 


0.62 


Hong Kong, SAR 


0.95 


0.85 


0.84 


Hungary 


0.95 


0.85 


0.84 


Indonesia 


0.94 


0.76 


0.77 


Iran, Islamic Rep. of 


0.92 


0.77 


0.78 


Israel 


0.95 


0.83 


0.81 


Italy 


0.95 


0.80 


0.79 


Japan 


0.96 


0.86 


0.86 


Jordan 


0.94 


0.84 


0.82 


Korea, Rep. of 


0.96 


0.86 


0.85 


Latvia 


0.94 


0.82 


0.82 


Lebanon 


0.91 


0.67 


0.68 


Lithuania 


0.95 


0.81 


0.81 


Macedonia, Rep. of 


0.95 


0.78 


0.82 


Malaysia 


0.96 


0.81 


0.83 


Moldova, Rep. of 


0.95 


0.79 


0.79 


Morocco 


0.86 


0.65 


0.71 


Netherlands 


0.94 


0.81 


0.84 


New Zealand 


0.94 


0.79 


0.79 


Norway 


0.93 


0.82 


0.81 


Palestinian Nat'l Auth. 


0.93 


0.81 


0.79 


Philippines 


0.93 


0.79 


0.80 


Romania 


0.95 


0.82 


0.81 


Russian Federation 


0.94 


0.77 


0.78 


Saudi Arabia 


0.80 


0.66 


0.65 


Scotland 


0.95 


0.82 


0.81 


Serbia 


0.95 


0.84 


0.84 


Singapore 


0.96 


0.88 


0.86 


Slovak Republic 


0.95 


0.85 


0.85 


Slovenia 


0.94 


0.77 


0.78 


South Africa 


0.89 


0.76 


0.78 


Sweden 


0.93 


0.76 


0.74 


Tunisia 


0.87 


0.65 


0.68 


United States 


0.97 


0.86 


0.85 


International Median 


0.95 


0.81 


0.81 


Benchmark Participants: 


Basque Country, Spain 


0.93 


0.78 


0.75 


Ontario Province, Can. 


0.92 


0.75 


0.76 


Quebec Province, Can. 


0.92 


0.75 


0.76 


Indiana State, US 


0.93 


0.82 


0.80 
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Exhibit B.4: Correlations of Mathematics Cognitive Domains 



MATHEMATICS 

Grade 




Countries 


Pearson Correlations for 
Mathematics Cognitive Domains 






Knowing 

Reasoning 


Applying 

Reasoning 


Armenia 


0.84 


0.74 


0.86 


Australia 


0.92 


0.89 


0.91 


Belgium (Flemish) 


0.89 


0.80 


0.84 


Chinese Taipei 


0.92 


0.87 


0.91 


Cyprus 


0.92 


0.87 


0.91 


England 


0.94 


0.89 


0.91 


Hong Kong, SAR 


0.91 


0.85 


0.90 


Hungary 


0.90 


0.82 


0.89 


Iran, Islamic Rep. of 


0.86 


0.73 


0.81 


Italy 


0.92 


0.84 


0.88 


Japan 


0.91 


0.84 


0.89 


Latvia 


0.91 


0.85 


0.88 


Lithuania 


0.93 


0.86 


0.90 


Moldova, Rep. of 


0.89 


0.82 


0.89 


Morocco 


0.80 


0.63 


0.74 


Netherlands 


0.87 


0.80 


0.85 


New Zealand 


0.93 


0.88 


0.90 


Norway 


0.92 


0.80 


0.86 


Philippines 


0.90 


0.83 


0.86 


Russian Federation 


0.88 


0.85 


0.90 


Scotland 


0.91 


0.85 


0.87 


Singapore 


0.92 


0.86 


0.94 


Slovenia 


0.91 


0.87 


0.92 


Tunisia 


0.80 


0.69 


0.73 


United States 


0.93 


0.88 


0.92 


International Median 


0.91 


0.85 


0.89 


Benchmark Participants: 


Ontario Province, Can. 


0.91 


0.87 


0.90 


Quebec Province, Can. 


0.91 


0.83 


0.86 


Indiana State, US 


0.90 


0.83 


0.88 
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medians of .95 between the knowing and applying domains, .81 
between the knowing and reasoning domains, and .81 between the 
applying and reasoning domains. The highest correlation was between 
the knowing and applying domains, which makes sense considering 
that these were the two domains with the highest correlation with 
mathematics achievement overall. 

At the fourth grade, country-level correlations between the 
cognitive domains also were high, with international medians of .91 
between the knowing and applying domains, .85 between the knowing 
and reasoning domains, and .89 between the applying and reason- 
ing domains. The relatively large correlations between the cognitive 
domain scales show that student performance in the cognitive domains 
is not independent, and that high-scoring students on one scale are 
likely also to be high scorers on another. Despite the high correla- 
tions, however, there is scope for interesting average score differences 
between countries on the three cognitive scales. 
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Exhibit C.1: Coverage of TIMSS 2003 Target Populations 



MATHEMATICS 

Grade 



o 

O 





i 


nternational Desired Population 


National Desired Population 


Countries 












Coverage 


Notes on Coverage 


School-Level 

Exclusions 


Within-Sample 

Exclusions 


Overall 

Exclusions 


Armenia 


100% 


2.9% 


0.0% 


2.9% 


Australia 


100% 


0.4% 


0.9% 


1.3% 


Bahrain 


100% 


0.0% 


0.0% 


0.0% 


Belgium (Flemish) 


100% 


3.1% 


0.1% 


3.2% 


Botswana 


100% 


0.8% 


2.2% 


3.0% 


Bulgaria 


100% 


0.5% 


0.0% 


0.5% 


Chile 


100% 


1 .6% 


0.7% 


2.2% 


Chinese Taipei 


100% 


0.2% 


4.6% 


4.8% 


Cyprus 


100% 


1.1% 


1 .5% 


2.5% 


Egypt 


100% 


3.4% 


0.0% 


3.4% 


England 


100% 


2.1% 


0.0% 


2.1% 


Estonia 


100% 


2.6% 


0.8% 


3.4% 


Ghana 


100% 


0.9% 


0.0% 


0.9% 


Hong Kong, SAR 


100% 


3.3% 


0.1% 


3.4% 


Hungary 


100% 


5.5% 


3.2% 


8.5% 


Indonesia 


80% 


Non-lslamic schools 


0.1% 


0.3% 


0.4% 


Iran, Islamic Rep. of 


100% 


5.5% 


1.1% 


6.5% 


Israel 


100% 


15.2% 


8.6% 


22.5% 


Italy 


100% 


0.0% 


3.6% 


3.6% 


Japan 


100% 


0.5% 


0.1% 


0.6% 


Jordan 


100% 


0.5% 


0.8% 


1 .3% 


Korea, Rep. of 


100% 


1 .5% 


3.4% 


4.9% 


Latvia 


100% 


3.6% 


0.1% 


3.7% 


Lebanon 


100% 


1 .4% 


0.0% 


1 .4% 


Lithuania 


89% 


Students taught in Lithuanian 


1 .4% 


1 .2% 


2.6% 


Macedonia, Rep. of 


100% 


12.5% 


0.0% 


12.5% 


Malaysia 


100% 


4.0% 


0.0% 


4.0% 


Moldova, Rep. of 


100% 


0.7% 


0.5% 


1 .2% 


Morocco 


69% 


All students but Souss Massa Draa, Casablanca, Gharb-Chrarda 


1 .5% 


0.0% 


1 .5% 


Netherlands 


100% 


3.0% 


0.0% 


3.0% 


New Zealand 


100% 


1 .7% 


2.7% 


4.4% 


Norway 


100% 


0.9% 


1 .5% 


2.3% 


Palestinian Nat 1 1 Auth. 


100% 


0.2% 


0.3% 


0.5% 


Philippines 


100% 


1 .5% 


0.0% 


1 .5% 


Romania 


100% 


0.4% 


0.1% 


0.5% 


Russian Federation 


100% 


1 .7% 


3.9% 


5.5% 


Saudi Arabia 


100% 


0.3% 


0.2% 


0.5% 


Scotland 


100% 


0.0% 


0.0% 


0.0% 


Serbia 


81% 


Serbia without Kosovo 


2.4% 


0.6% 


2.9% 


Singapore 


100% 


0.0% 


0.0% 


0.0% 


Slovak Republic 


100% 


5.0% 


0.0% 


5.0% 


Slovenia 


100% 


1 .3% 


0.1% 


1 .4% 


South Africa 


100% 


0.6% 


0.0% 


0.6% 


Sweden 


100% 


0.3% 


2.5% 


2.8% 


Syrian Arab Republic 


100% 


18.7% 


0.0% 


18.8% 


Tunisia 


100% 


1 .8% 


0.0% 


1 .8% 


United States 


100% 


0.0% 


4.9% 


4.9% 


Benchmarking Participants 


Basque Region, Spain 


100% 


2.1% 


3.8% 


5.8% 


Indiana State, US 


100% 


0.0% 


7.8% 


7.8% 


Ontario Province, Can. 


100% 


1 .0% 


5.0% 


6.0% 


Quebec Province, Can. 


100% 


1 .4% 


3.5% 


4.8% 
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Exhibit C.1: Coverage of TIMSS 2003 Target Populations 



MATHEMATICS 

Grade 





i 


nternational Desired Population 


National Desired Population 


Countries 












Coverage 


Notes on Coverage 


School-Level 

Exclusions 


Within-Sample 

Exclusions 


Overall 

Exclusions 


Armenia 


100% 


2.9% 


0.0% 


2.9% 


Australia 


100% 


1.2% 


1.6% 


2.7% 


Belgium (Flemish) 


100% 


5.9% 


0.4% 


6.3% 


Chinese Taipei 


100% 


0.3% 


2.8% 


3.1% 


Cyprus 


100% 


1.5% 


1.4% 


2.9% 


England 


100% 


1.9% 


0.0% 


1.9% 


Hong Kong, SAR 


100% 


3.7% 


0.1% 


3.8% 


Hungary 


100% 


4.4% 


3.9% 


8.1% 


Iran, Islamic Rep. of 


100% 


3.6% 


2.1% 


5.7% 


Italy 


100% 


0.1% 


4.1% 


4.2% 


Japan 


100% 


0.4% 


0.3% 


0.8% 


Latvia 


100% 


4.3% 


0.1% 


4.4% 


Lithuania 


92% 


Students taught in Lithuanian 


2.1% 


2.6% 


4.6% 


Moldova, Rep. of 


100% 


2.0% 


1.6% 


3.6% 


Morocco 


100% 


2.2% 


0.0% 


2.2% 


Netherlands 


100% 


4.1% 


1.1% 


5.2% 


New Zealand 


100% 


1.5% 


2.5% 


4.0% 


Norway 


100% 


1.7% 


2.7% 


4.4% 


Philippines 


100% 


3.8% 


0.7% 


4.5% 


Russian Federation 


100% 


2.2% 


4.7% 


6.8% 


Scotland 


100% 


1.5% 


0.0% 


1.5% 


Singapore 


100% 


0.0% 


0.0% 


0.0% 


Slovenia 


100% 


0.8% 


0.5% 


1.3% 


Tunisia 


100% 


0.9% 


0.0% 


0.9% 


United States 


100% 


0.0% 


5.1% 


5.1% 


Yemen 


100% 


0.6% 


8.9% 


9.5% 


Benchmarking Participants 










Indiana State, US 


100% 


0.0% 


7.2% 


7.2% 


Ontario Province, Can. 


100% 


1.3% 


3.5% 


4.8% 


Quebec Province, Can. 


100% 


2.7% 


0.9% 


3.6% 
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Exhibit C.2: Participation Rates (Weighted) 



MATHEMATICS 

Grade 



o 

O 





School Participation 


Class 


Student 


Overall Participation 


Countries 


Before 

Replacement 


After 

Replacement 


Participation 


Participation 


Before 

Replacement 


After 

Replacement 


Armenia 


99% 


99% 


99% 


90% 


89% 


89% 


Australia 


81% 


90% 


100% 


93% 


75% 


83% 


Bahrain 


100% 


100% 


100% 


98% 


98% 


98% 


Belgium (Flemish) 


82% 


99% 


98% 


97% 


77% 


94% 


Botswana 


98% 


98% 


100% 


98% 


96% 


96% 


Bulgaria 


97% 


97% 


99% 


96% 


92% 


92% 


Chile 


98% 


100% 


100% 


99% 


97% 


99% 


Chinese Taipei 


100% 


100% 


100% 


99% 


99% 


99% 


Cyprus 


100% 


100% 


100% 


96% 


96% 


96% 


Egypt 


99% 


100% 


100% 


97% 


97% 


97% 


England 


40% 


54% 


99% 


86% 


34% 


46% 


Estonia 


99% 


99% 


100% 


96% 


95% 


95% 


Ghana 


100% 


100% 


100% 


93% 


93% 


93% 


Hong Kong, SAR 


74% 


83% 


99% 


97% 


72% 


80% 


Hungary 


98% 


99% 


100% 


95% 


94% 


94% 


Indonesia 


98% 


100% 


100% 


99% 


97% 


99% 


Iran, Islamic Rep. of 


100% 


100% 


100% 


98% 


98% 


98% 


Israel 


98% 


99% 


100% 


95% 


93% 


94% 


Italy 


96% 


100% 


100% 


97% 


93% 


97% 


Japan 


97% 


97% 


100% 


96% 


93% 


93% 


Jordan 


100% 


100% 


100% 


96% 


96% 


96% 


Korea, Rep. of 


99% 


99% 


100% 


99% 


98% 


98% 


Latvia 


92% 


94% 


100% 


89% 


81% 


83% 


Lebanon 


93% 


95% 


100% 


96% 


89% 


91% 


Lithuania 


92% 


95% 


100% 


89% 


81% 


84% 


Macedonia, Rep. of 


94% 


99% 


100% 


97% 


91% 


96% 


Malaysia 


100% 


100% 


100% 


98% 


98% 


98% 


Moldova, Rep. of 


99% 


100% 


100% 


96% 


95% 


96% 


Morocco 


79% 


79% 


100% 


91% 


71% 


71% 


Netherlands 


79% 


87% 


100% 


94% 


74% 


81% 


New Zealand 


86% 


97% 


100% 


93% 


80% 


90% 


Norway 


92% 


92% 


100% 


92% 


85% 


85% 


Palestinian Nat’l Auth. 


100% 


100% 


100% 


99% 


99% 


99% 


Philippines 


81% 


86% 


100% 


96% 


78% 


82% 


Romania 


99% 


99% 


100% 


98% 


98% 


98% 


Russian Federation 


99% 


99% 


100% 


97% 


96% 


96% 


Saudi Arabia 


95% 


97% 


100% 


97% 


93% 


94% 


Scotland 


76% 


85% 


100% 


89% 


68% 


76% 


Serbia 


99% 


99% 


100% 


96% 


96% 


96% 


Singapore 


100% 


100% 


100% 


97% 


97% 


97% 


Slovak Republic 


96% 


100% 


100% 


95% 


91% 


95% 


Slovenia 


94% 


99% 


100% 


93% 


87% 


91% 


South Africa 


89% 


96% 


100% 


92% 


82% 


88% 


Sweden 


97% 


99% 


99% 


89% 


85% 


87% 


Syrian Arab Republic 


81% 


89% 


100% 


98% 


79% 


87% 


Tunisia 


100% 


100% 


100% 


98% 


98% 


98% 


United States 


71% 


78% 


99% 


94% 


66% 


73% 


Benchmarking Participants 


Basque Region, Spain 


100% 


100% 


100% 


98% 


97% 


98% 


Indiana State, US 


97% 


97% 


100% 


97% 


94% 


94% 


Ontario Province, Can. 


84% 


93% 


100% 


95% 


80% 


89% 


Quebec Province, Can. 


91% 


93% 


100% 


92% 


84% 


85% 
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Exhibit C.2: Participation Rates (Weighted) 



MATHEMATICS 

Grade 



Countries 


School Participation 


Class 


Student 


Overall Participation 


Before 

Replacement 


After 

Replacement 


Participation 


Participation 


Before 

Replacement 


After 

Replacement 


Armenia 


99% 


99% 


100% 


91% 


90% 


90% 


Australia 


78% 


90% 


100% 


94% 


73% 


85% 


Belgium (Flemish) 


89% 


99% 


100% 


98% 


87% 


97% 


Chinese Taipei 


100% 


100% 


100% 


99% 


99% 


99% 


Cyprus 


100% 


100% 


100% 


97% 


97% 


97% 


England 


54% 


82% 


100% 


93% 


50% 


76% 


Hong Kong, SAR 


77% 


88% 


99% 


95% 


73% 


83% 


Hungary 


98% 


99% 


100% 


94% 


92% 


93% 


Iran, Islamic Rep. of 


100% 


100% 


100% 


98% 


98% 


98% 


Italy 


97% 


100% 


100% 


97% 


93% 


97% 


Japan 


100% 


100% 


100% 


97% 


97% 


97% 


Latvia 


91% 


94% 


100% 


94% 


85% 


88% 


Lithuania 


92% 


96% 


99% 


92% 


84% 


87% 


Moldova, Rep. of 


97% 


100% 


100% 


97% 


94% 


97% 


Morocco 


87% 


87% 


100% 


93% 


81% 


81% 


Netherlands 


52% 


87% 


100% 


96% 


50% 


84% 


New Zealand 


87% 


98% 


100% 


95% 


82% 


93% 


Norway 


89% 


93% 


100% 


95% 


85% 


88% 


Philippines 


78% 


85% 


100% 


95% 


75% 


81% 


Russian Federation 


99% 


100% 


100% 


97% 


96% 


97% 


Scotland 


64% 


83% 


100% 


92% 


59% 


77% 


Singapore 


100% 


100% 


100% 


98% 


98% 


98% 


Slovenia 


95% 


99% 


100% 


92% 


87% 


91% 


Tunisia 


100% 


100% 


100% 


99% 


99% 


99% 


United States 


70% 


82% 


99% 


95% 


66% 


78% 


Yemen 


100% 


100% 


100% 


93% 


93% 


93% 


Benchmarking Participants 














Indiana State, US 


100% 


100% 


100% 


98% 


98% 


98% 


Ontario Province, Can. 


89% 


94% 


100% 


96% 


85% 


90% 


Quebec Province, Can. 


99% 


100% 


100% 


91% 


90% 


91% 
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APPENDIX D: PERCENTILES AND STANDARD DEVIATIONS OF MATHEMATICS ACHIEVEMENT IN THE COGNITIVE DOMAINS 
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Exhibit D.1: Percentiles of Achievement in Knowing Cognitive Domain 



MATHEMATICS 

Grade 



o 

O 



Countries 


5th 

Percentile 


25th 

Percentile 


50th 

Percentile 


75th 

Percentile 


95th 

Percentile 


Armenia 


347 (4.9) 


429 (3.4) 


484 (2.9) 


535 (4.2) 


600 (3.7) 


Australia 


381 (7.8) 


451 (3.2) 


497 (5.2) 


544 (4.8) 


609 (7.8) 


Bahrain 


265 (4.9) 


342 (2.4) 


401 (3.1) 


459 (2.6) 


538 (5.5) 


Belgium (Flemish) 


405 (6.9) 


499 (3.6) 


546 (2.6) 


585 (3.5) 


634 (4.9) 


Botswana 


251 (5.0) 


321 (2.9) 


373 (2.3) 


423 (3.4) 


492 (5.4) 


Bulgaria 


352 (8.5) 


431 (4.9) 


486 (5.2) 


542 (5.2) 


618 (5.8) 


Chile 


268 (3.8) 


332 (3.2) 


380 (3.3) 


435 (4.7) 


523 (5.2) 


Chinese Taipei 


411 (6.0) 


520 (5.8) 


595 (4.5) 


656 (4.5) 


731 (4.8) 


Cyprus 


347 (5.5) 


418 (2.5) 


469 (2.3) 


515 (2.0) 


574 (2.7) 


Egypt 


263 (8.8) 


346 (4.6) 


409 (3.5) 


475 (4.5) 


564 (4.4) 


England 


390 (6.8) 


445 (5.8) 


486 (4.9) 


532 (7.3) 


593 (6.9) 


Estonia 


434 (5.5) 


495 (4.7) 


538 (3.4) 


581 (4.1) 


641 (5.7) 


Ghana 


43 (5.2) 


152 (4.9) 


229 (5.8) 


310 (8.9) 


427 (8.7) 


Hong Kong, SAR 


467 (9.9) 


549 (5.3) 


594 (4.3) 


636 (3.2) 


689 (4.0) 


Hungary 


416 (4.8) 


487 (4.5) 


538 (3.3) 


587 (2.7) 


654 (4.9) 


Indonesia 


284 (8.2) 


364 (6.3) 


421 (4.8) 


480 (4.9) 


560 (4.3) 


Iran, Islamic Rep. of 


286 (2.9) 


352 (2.1) 


401 (2.7) 


456 (3.4) 


533 (3.0) 


Israel 


374 (4.8) 


450 (4.2) 


503 (3.8) 


553 (3.1) 


622 (6.0) 


Italy 


358 (5.0) 


435 (3.6) 


487 (3.0) 


535 (3.3) 


602 (7.5) 


Japan 


443 (3.2) 


519 (3.8) 


567 (2.1) 


611 (1.7) 


677 (6.6) 


Jordan 


273 (6.0) 


364 (5.1) 


429 (6.0) 


494 (4.9) 


580 (8.7) 


Korea, Rep. of 


444 (7.0) 


539 (3.9) 


599 (2.8) 


650 (2.7) 


717 (4.4) 


Latvia 


408 (4.9) 


474 (4.4) 


520 (3.2) 


564 (3.0) 


625 (5.2) 


Lebanon 


333 (6.0) 


398 (3.8) 


447 (3.9) 


496 (4.5) 


563 (4.2) 


Lithuania 


390 (6.4) 


462 (3.3) 


513 (3.7) 


561 (3.2) 


627 (4.0) 


Macedonia, Rep. of 


298 (8.8) 


388 (4.3) 


450 (5.6) 


508 (3.6) 


584 (5.8) 


Malaysia 


393 (5.8) 


456 (3.8) 


505 (4.8) 


556 (5.1) 


619 (5.0) 


Moldova, Rep. of 


325 (5.7) 


411 (5.6) 


471 (4.3) 


525 (6.0) 


593 (4.9) 


Morocco 


260 (8.1) 


334 (4.9) 


386 (4.9) 


438 (4.5) 


512 (8.4) 


Netherlands 


422 (3.8) 


480 (4.3) 


522 (4.3) 


563 (3.3) 


611 (5.4) 


New Zealand 


377 (7.7) 


440 (3.7) 


484 (4.7) 


529 (7.9) 


596 (14.3) 


Norway 


352 (3.4) 


413 (2.2) 


453 (2.5) 


490 (2.5) 


538 (2.3) 


Palestinian Nat'l Auth. 


228 (5.1) 


320 (4.0) 


390 (4.6) 


462 (4.5) 


554 (5.1) 


Philippines 


249 (4.7) 


326 (3.6) 


383 (6.0) 


448 (6.9) 


538 (7.4) 


Romania 


327 (8.6) 


421 (6.4) 


491 (7.1) 


553 (5.3) 


629 (7.0) 


Russian Federation 


395 (7.4) 


469 (3.5) 


520 (3.2) 


570 (3.8) 


640 (5.5) 


Saudi Arabia 


170 (8.7) 


256 (5.8) 


312 (5.7) 


373 (4.9) 


465 (9.0) 


Scotland 


377 (4.2) 


441 (4.0) 


483 (3.5) 


524 (3.4) 


575 (5.6) 


Serbia 


357 (4.4) 


441 (3.1) 


499 (4.3) 


554 (2.8) 


620 (3.5) 


Singapore 


461 (5.3) 


546 (4.8) 


598 (3.9) 


642 (1.9) 


696 (3.2) 


Slovak Republic 


388 (8.3) 


467 (4.0) 


519 (4.4) 


569 (3.5) 


637 (4.7) 


Slovenia 


391 (5.4) 


455 (2.0) 


499 (2.4) 


543 (2.1) 


606 (7.3) 


South Africa 


116 (4.1) 


190 (3.2) 


247 (4.3) 


313 (7.1) 


477 (22.1) 


Sweden 


392 (4.0) 


450 (3.7) 


487 (2.2) 


523 (1.5) 


575 (3.1) 


Tunisia 


290 (4.7) 


351 (2.8) 


396 (3.1) 


444 (3.2) 


517 (6.4) 


United States 


397 (3.2) 


462 (3.6) 


509 (3.2) 


557 (3.3) 


623 (3.9) 


Benchmarking Participants 












Basque Country, Spain 


395 (8.4) 


456 (3.2) 


497 (2.2) 


536 (1.8) 


589 (4.6) 


Indiana State, US 


415 (6.8) 


473 (4.1) 


514 (5.1) 


556 (6.6) 


618 (9.6) 


Ontario Province, Can. 


419 (5.4) 


473 (3.7) 


513 (2.4) 


552 (3.0) 


605 (4.2) 


Quebec Province, Can. 


452 (2.9) 


500 (2.7) 


537 (3.1) 


574 (4.4) 


624 (4.4) 



( ) Standard errors appear in parentheses. 
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Exhibit D.1: Percentiles of Achievement in Knowing Cognitive Domain 



MATHEMATICS 

Grade 




Countries 


5th 

Percentile 


25th 

Percentile 


50th 

Percentile 


75th 

Percentile 


95th 

Percentile 


Armenia 


296 (7.6) 


387 (5.9) 


447 (5.4) 


507 (5.0) 


597 (4.6) 


Australia 


363 (8.0) 


449 (5.7) 


505 (4.6) 


554 (3.3) 


631 (5.7) 


Belgium (Flemish) 


450 (3.4) 


512 (2.3) 


558 (2.3) 


604 (3.7) 


665 (2.9) 


Chinese Taipei 


443 (4.5) 


518 (2.6) 


566 (2.2) 


614 (3.1) 


681 (4.1) 


Cyprus 


351 (3.9) 


443 (4.6) 


503 (4.3) 


560 (2.4) 


641 (5.6) 


England 


378 (7.7) 


472 (4.4) 


536 (5.1) 


599 (5.0) 


687 (4.5) 


Hong Kong, SAR 


458 (4.5) 


528 (3.6) 


576 (3.6) 


620 (4.0) 


683 (3.7) 


Hungary 


382 (4.2) 


464 (3.2) 


520 (3.5) 


572 (4.8) 


644 (5.7) 


Iran, Islamic Rep. of 


275 (8.3) 


351 (3.2) 


403 (3.3) 


457 (5.0) 


533 (3.1) 


Italy 


365 (5.4) 


456 (3.0) 


514 (3.8) 


573 (4.2) 


660 (6.5) 


Japan 


424 (4.2) 


512 (3.7) 


566 (3.0) 


620 (2.7) 


696 (4.5) 


Latvia 


403 (6.6) 


473 (4.0) 


519 (2.8) 


562 (2.9) 


623 (5.2) 


Lithuania 


392 (5.0) 


469 (3.6) 


522 (2.9) 


571 (3.1) 


642 (4.7) 


Moldova, Rep. of 


337 (11.3) 


437 (5.0) 


504 (5.5) 


566 (6.0) 


654 (9.3) 


Morocco 


217 (5.2) 


299 (6.2) 


360 (4.6) 


420 (5.0) 


500 (7.4) 


Netherlands 


440 (3.5) 


494 (2.9) 


531 (2.4) 


566 (1.9) 


617 (3.1) 


New Zealand 


349 (5.1) 


437 (3.5) 


494 (3.0) 


551 (2.7) 


629 (4.3) 


Norway 


315 (4.6) 


398 (2.8) 


451 (2.6) 


501 (3.2) 


568 (4.2) 


Philippines 


231 (6.6) 


315 (4.5) 


378 (3.9) 


449 (8.9) 


559 (14.3) 


Russian Federation 


381 (5.9) 


455 (5.8) 


511 (6.7) 


570 (5.1) 


655 (8.1) 


Scotland 


356 (3.0) 


434 (4.1) 


485 (3.5) 


535 (5.6) 


605 (5.7) 


Singapore 


442 (12.5) 


563 (9.2) 


633 (7.4) 


697 (7.2) 


784 (6.7) 


Slovenia 


351 (5.4) 


424 (3.4) 


473 (2.3) 


517 (3.8) 


580 (3.3) 


Tunisia 


188 (7.8) 


275 (5.3) 


338 (6.0) 


400 (5.6) 


487 (3.5) 


United States 


396 (4.1) 


474 (2.7) 


529 (2.3) 


584 (3.5) 


657 (3.7) 


Benchmarking Participants 


Indiana State, US 


425 (5.8) 


497 (6.0) 


544 (4.5) 


591 (3.8) 


661 (7.9) 


Ontario Province, Can. 


392 (5.3) 


465 (4.2) 


513 (3.3) 


562 (4.4) 


635 (8.7) 


Quebec Province, Can. 


392 (5.0) 


459 (2.9) 


504 (3.6) 


551 (3.5) 


613 (7.6) 



( ) Standard errors appear in parentheses. 
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Exhibit D.2: Percentiles of Achievement in Applying Cognitive Domain 



MATHEMATICS 

Grade 




Countries 


5th 

Percentile 


25th 

Percentile 


50th 

Percentile 


75th 

Percentile 


95th 

Percentile 


Armenia 


326 (4.5) 


421 (3.8) 


483 (4.1) 


541 (3.7) 


607 (2.2) 


Australia 


365 (7.6) 


452 (5.4) 


510 (5.4) 


567 (6.2) 


645 (6.8) 


Bahrain 


272 (2.0) 


345 (2.9) 


398 (2.6) 


451 (2.5) 


522 (3.7) 


Belgium (Flemish) 


402 (6.2) 


494 (3.3) 


543 (3.0) 


587 (3.3) 


642 (3.6) 


Botswana 


256 (4.9) 


320 (3.7) 


367 (3.0) 


416 (2.9) 


486 (6.3) 


Bulgaria 


328 (6.4) 


412 (7.6) 


471 (3.7) 


531 (5.0) 


614 (7.0) 


Chile 


261 (4.9) 


332 (3.5) 


386 (3.6) 


446 (5.5) 


537 (7.0) 


Chinese Taipei 


400 (7.3) 


513 (4.8) 


594 (4.3) 


655 (4.3) 


733 (4.5) 


Cyprus 


309 (3.5) 


396 (5.1) 


461 (2.8) 


521 (2.4) 


595 (3.0) 


Egypt 


251 (5.9) 


336 (3.7) 


401 (5.0) 


470 (3.7) 


565 (5.9) 


England 


377 (6.9) 


448 (6.1) 


500 (6.4) 


559 (7.0) 


636 (8.3) 


Estonia 


412 (4.1) 


481 (2.7) 


527 (3.4) 


576 (2.3) 


646 (5.4) 


Ghana 


162 (4.6) 


237 (4.9) 


292 (4.1) 


348 (6.6) 


430 (13.0) 


Hong Kong, SAR 


448 (15.6) 


544 (4.6) 


591 (3.1) 


634 (3.1) 


690 (2.7) 


Hungary 


389 (8.7) 


466 (4.2) 


524 (4.2) 


581 (4.0) 


654 (7.0) 


Indonesia 


262 (6.0) 


344 (6.9) 


405 (5.7) 


471 (6.3) 


567 (7.1) 


Iran, Islamic Rep. of 


296 (3.8) 


362 (2.7) 


413 (2.9) 


467 (3.5) 


546 (5.5) 


Israel 


344 (6.4) 


433 (5.9) 


497 (4.1) 


558 (3.9) 


640 (5.3) 


Italy 


349 (6.1) 


431 (3.1) 


486 (3.2) 


539 (3.2) 


610 (5.6) 


Japan 


423 (3.2) 


512 (3.1) 


567 (2.5) 


618 (2.5) 


694 (7.5) 


Jordan 


274 (5.6) 


360 (5.4) 


422 (5.6) 


484 (6.0) 


568 (6.2) 


Korea, Rep. of 


434 (2.5) 


531 (2.3) 


591 (2.7) 


643 (2.6) 


711 (4.5) 


Latvia 


376 (4.7) 


452 (3.5) 


506 (3.5) 


558 (5.8) 


626 (4.8) 


Lebanon 


312 (3.6) 


376 (2.9) 


424 (4.0) 


475 (4.9) 


546 (4.4) 


Lithuania 


361 (3.3) 


442 (3.9) 


499 (3.9) 


557 (3.9) 


635 (4.2) 


Macedonia, Rep. of 


277 (5.8) 


368 (6.7) 


431 (3.9) 


491 (4.3) 


571 (12.6) 


Malaysia 


386 (5.2) 


456 (4.1) 


511 (6.1) 


569 (4.6) 


638 (6.5) 


Moldova, Rep. of 


321 (6.0) 


402 (3.9) 


462 (5.0) 


516 (3.2) 


582 (4.3) 


Morocco 


278 (3.8) 


339 (2.5) 


383 (2.7) 


428 (3.4) 


495 (5.7) 


Netherlands 


427 (7.1) 


495 (6.0) 


545 (4.9) 


593 (5.3) 


651 (4.4) 


New Zealand 


365 (8.2) 


441 (6.8) 


497 (5.2) 


551 (6.5) 


630 (7.1) 


Norway 


341 (6.6) 


420 (4.2) 


472 (2.6) 


521 (2.7) 


583 (4.4) 


Palestinian Nat 1 1 Auth. 


242 (5.3) 


325 (2.6) 


387 (2.5) 


453 (3.6) 


538 (3.1) 


Philippines 


250 (5.1) 


320 (5.9) 


373 (4.9) 


433 (6.8) 


524 (11.3) 


Romania 


321 (7.7) 


411 (4.5) 


477 (5.5) 


540 (6.6) 


621 (6.6) 


Russian Federation 


371 (5.0) 


448 (4.7) 


504 (3.9) 


560 (3.5) 


634 (5.8) 


Saudi Arabia 


216 (8.5) 


289 (5.2) 


338 (4.1) 


388 (4.7) 


463 (4.7) 


Scotland 


373 (4.8) 


455 (6.5) 


509 (4.5) 


560 (5.8) 


624 (5.0) 


Serbia 


304 (4.1) 


401 (4.7) 


468 (3.8) 


537 (4.2) 


622 (2.4) 


Singapore 


461 (7.7) 


562 (5.7) 


621 (3.8) 


669 (2.8) 


725 (3.7) 


Slovak Republic 


359 (6.6) 


443 (3.4) 


502 (5.4) 


562 (4.4) 


645 (4.2) 


Slovenia 


370 (5.7) 


440 (2.5) 


491 (2.7) 


542 (2.9) 


614 (4.1) 


South Africa 


125 (4.1) 


199 (3.2) 


254 (4.6) 


320 (7.4) 


486 (23.7) 


Sweden 


374 (6.7) 


455 (4.7) 


507 (2.6) 


557 (3.4) 


627 (5.5) 


Tunisia 


321 (2.4) 


376 (3.1) 


416 (2.5) 


459 (2.6) 


526 (5.0) 


United States 


364 (4.2) 


444 (3.4) 


502 (4.4) 


560 (3.5) 


639 (4.6) 


Benchmarking Participants 












Basque Country, Spain 


370 (3.6) 


437 (2.9) 


483 (4.0) 


528 (2.8) 


588 (4.3) 


Indiana State, US 


382 (10.9) 


456 (6.6) 


506 (5.5) 


559 (6.4) 


634 (9.8) 


Ontario Province, Can. 


405 (3.2) 


474 (3.9) 


524 (3.5) 


572 (3.5) 


636 (3.1) 


Quebec Province, Can. 


450 (2.1) 


504 (4.4) 


544 (4.1) 


585 (5.3) 


640 (7.3) 



( ) Standard errors appear in parentheses. 
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TIMSS2003 



Exhibit D.2: Percentiles of Achievement in Applying Cognitive Domain 



MATHEMATICS 

Grade 




Countries 


5th 

Percentile 


25th 

Percentile 


50th 

Percentile 


75th 

Percentile 


95th 

Percentile 


Armenia 


326 (5.5) 


407 (5.0) 


461 (2.9) 


516 (4.7) 


598 (6.9) 


Australia 


350 (6.9) 


434 (4.9) 


491 (5.3) 


545 (6.4) 


626 (6.7) 


Belgium (Flemish) 


447 (4.6) 


505 (1.7) 


546 (2.4) 


588 (1.9) 


643 (4.0) 


Chinese Taipei 


458 (4.7) 


523 (2.1) 


563 (2.6) 


602 (2.3) 


656 (3.8) 


Cyprus 


365 (5.2) 


455 (2.7) 


512 (3.3) 


568 (2.6) 


647 (4.3) 


England 


370 (8.0) 


463 (3.9) 


527 (4.7) 


591 (6.3) 


677 (4.5) 


Hong Kong, SAR 


463 (5.1) 


533 (3.9) 


579 (2.4) 


623 (3.8) 


681 (3.6) 


Hungary 


396 (4.8) 


478 (4.3) 


533 (3.6) 


586 (4.3) 


655 (5.9) 


Iran, Islamic Rep. of 


261 (5.6) 


336 (5.3) 


391 (4.5) 


444 (4.2) 


519 (6.7) 


Italy 


355 (4.5) 


438 (4.2) 


494 (3.4) 


549 (5.0) 


631 (7.6) 


Japan 


430 (4.8) 


513 (1.9) 


567 (2.0) 


622 (2.2) 


697 (3.8) 


Latvia 


409 (5.8) 


493 (4.0) 


549 (2.9) 


600 (4.2) 


673 (4.5) 


Lithuania 


414 (6.1) 


491 (4.0) 


545 (3.2) 


595 (2.6) 


664 (4.1) 


Moldova, Rep. of 


355 (9.8) 


448 (5.2) 


510 (3.4) 


569 (4.7) 


647 (6.4) 


Morocco 


212 (7.8) 


291 (5.8) 


349 (5.5) 


408 (4.5) 


484 (5.3) 


Netherlands 


442 (8.2) 


503 (2.7) 


542 (2.4) 


582 (3.6) 


636 (3.3) 


New Zealand 


341 (3.6) 


427 (5.0) 


488 (1.3) 


546 (2.3) 


624 (7.6) 


Norway 


318 (4.7) 


397 (3.3) 


449 (2.6) 


498 (3.2) 


567 (2.4) 


Philippines 


211 (5.1) 


292 (5.0) 


356 (5.7) 


427 (8.9) 


542 (24.5) 


Russian Federation 


405 (7.8) 


484 (6.2) 


541 (4.9) 


599 (4.5) 


682 (5.5) 


Scotland 


355 (3.5) 


433 (2.6) 


488 (4.1) 


541 (3.3) 


617 (5.2) 


Singapore 


430 (12.1) 


540 (7.3) 


601 (6.5) 


658 (6.1) 


731 (6.3) 


Slovenia 


344 (5.3) 


426 (3.5) 


481 (5.4) 


531 (4.0) 


597 (5.7) 


Tunisia 


182 (6.9) 


276 (6.2) 


348 (7.6) 


420 (5.9) 


516 (6.3) 


United States 


378 (4.9) 


451 (3.8) 


506 (2.7) 


560 (2.7) 


631 (2.9) 


Benchmarking Participants 


Indiana State, US 


402 (6.7) 


475 (4.5) 


524 (3.5) 


570 (4.5) 


640 (5.9) 


Ontario Province, Can. 


378 (4.5) 


449 (2.8) 


497 (4.0) 


546 (5.6) 


621 (10.2) 


Quebec Province, Can. 


386 (3.6) 


454 (2.6) 


499 (3.0) 


543 (4.5) 


605 (4.2) 



( ) Standard errors appear in parentheses. 
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MATHEMATICS 

Grade 




Countries 


5th 

Percentile 


25th 

Percentile 


50th 

Percentile 


75th 

Percentile 


95th 

Percentile 


Armenia 


293 (5.9) 


402 (4.7) 


472 (5.2) 


541 (3.1) 


622 (2.6) 


Australia 


375 (5.5) 


461 (5.0) 


520 (5.6) 


572 (5.5) 


642 (6.3) 


Bahrain 


297 (2.8) 


374 (3.5) 


426 (2.7) 


477 (2.9) 


545 (5.9) 


Belgium (Flemish) 


395 (5.5) 


487 (3.7) 


539 (2.3) 


586 (3.3) 


648 (3.3) 


Botswana 


204 (3.6) 


292 (4.0) 


353 (4.0) 


415 (4.4) 


502 (3.9) 


Bulgaria 


325 (6.8) 


412 (4.4) 


472 (5.2) 


533 (4.7) 


610 (5.4) 


Chile 


261 (5.5) 


346 (5.0) 


408 (4.0) 


472 (4.3) 


561 (4.2) 


Chinese Taipei 


414 (6.7) 


514 (5.0) 


581 (4.0) 


642 (3.2) 


721 (4.2) 


Cyprus 


308 (3.2) 


399 (3.7) 


458 (2.6) 


516 (2.5) 


593 (6.3) 


Egypt 


247 (6.8) 


335 (3.4) 


401 (3.4) 


464 (5.2) 


554 (5.6) 


England 


373 (5.9) 


451 (6.6) 


509 (6.5) 


567 (6.5) 


643 (7.4) 


Estonia 


399 (7.9) 


472 (3.6) 


524 (2.8) 


574 (3.6) 


643 (4.8) 


Ghana 


149 (4.7) 


248 (4.0) 


314 (5.5) 


380 (5.9) 


471 (5.4) 


Hong Kong, SAR 


436 (9.1) 


522 (4.0) 


574 (4.2) 


621 (2.6) 


684 (3.5) 


Hungary 


402 (6.0) 


477 (3.6) 


530 (3.0) 


582 (4.7) 


655 (6.9) 


Indonesia 


253 (7.5) 


344 (5.5) 


406 (6.1) 


468 (4.7) 


556 (7.6) 


Iran, Islamic Rep. of 


295 (5.0) 


365 (3.2) 


417 (2.9) 


468 (2.7) 


540 (2.4) 


Israel 


328 (4.1) 


419 (4.1) 


485 (5.4) 


547 (3.8) 


632 (5.4) 


Italy 


360 (8.1) 


438 (4.1) 


491 (3.2) 


542 (3.3) 


610 (5.4) 


Japan 


446 (7.1) 


528 (3.1) 


577 (1.5) 


625 (3.3) 


698 (5.1) 


Jordan 


295 (6.1) 


377 (5.4) 


435 (4.1) 


490 (4.5) 


566 (6.6) 


Korea, Rep. of 


441 (3.6) 


530 (2.3) 


585 (1 .6) 


638 (2.3) 


712 (3.7) 


Latvia 


367 (6.4) 


447 (4.2) 


501 (3.4) 


553 (3.7) 


629 (5.7) 


Lebanon 


278 (6.9) 


355 (3.1) 


410 (3.4) 


464 (3.7) 


540 (4.4) 


Lithuania 


354 (5.1) 


435 (4.3) 


490 (2.6) 


545 (2.7) 


618 (3.2) 


Macedonia, Rep. of 


278 (8.3) 


376 (4.5) 


442 (4.9) 


504 (5.6) 


584 (4.6) 


Malaysia 


386 (2.9) 


453 (2.8) 


504 (4.9) 


554 (4.1) 


618 (3.9) 


Moldova, Rep. of 


309 (8.7) 


395 (6.2) 


456 (3.6) 


513 (3.6) 


590 (3.3) 


Morocco 


259 (7.4) 


338 (4.2) 


392 (3.2) 


446 (4.5) 


520 (8.2) 


Netherlands 


416 (6.2) 


490 (4.8) 


543 (4.0) 


594 (5.3) 


660 (8.3) 


New Zealand 


374 (5.2) 


455 (5.5) 


513 (6.8) 


565 (8.1) 


639 (7.0) 


Norway 


343 (6.0) 


427 (3.6) 


483 (3.0) 


534 (2.7) 


604 (3.3) 


Palestinian Nat'l Auth. 


259 (6.1) 


346 (3.3) 


405 (3.9) 


465 (2.7) 


545 (5.6) 


Philippines 


177 (6.1) 


282 (5.4) 


355 (7.4) 


433 (6.2) 


542 (8.0) 


Romania 


297 (8.7) 


395 (4.6) 


461 (5.1) 


523 (5.5) 


611 (4.8) 


Russian Federation 


365 (8.9) 


443 (4.7) 


497 (3.8) 


551 (4.4) 


623 (3.4) 


Saudi Arabia 


205 (7.9) 


289 (6.1) 


349 (3.9) 


407 (3.7) 


489 (6.5) 


Scotland 


375 (7.9) 


459 (4.0) 


518 (3.9) 


570 (4.1) 


638 (8.5) 


Serbia 


323 (6.0) 


411 (3.4) 


471 (3.5) 


527 (2.7) 


604 (5.5) 


Singapore 


424 (5.6) 


528 (4.8) 


591 (3.2) 


645 (2.9) 


717 (3.7) 


Slovak Republic 


360 (5.5) 


450 (7.2) 


506 (4.9) 


560 (3.9) 


636 (4.4) 


Slovenia 


372 (4.7) 


445 (2.8) 


495 (3.3) 


543 (1 .9) 


610 (3.8) 


South Africa 


111 (8.0) 


207 (3.6) 


277 (3.4) 


354 (7.1) 


505 (13.7) 


Sweden 


364 (9.3) 


453 (4.0) 


510 (2.9) 


567 (3.3) 


642 (5.9) 


Tunisia 


282 (4.9) 


351 (3.0) 


400 (3.3) 


447 (2.4) 


516 (7.7) 


United States 


366 (5.0) 


448 (4.1) 


507 (4.8) 


564 (2.9) 


638 (4.5) 


Benchmarking Participants 












Basque Country, Spain 


371 (7.9) 


446 (3.1) 


496 (2.3) 


545 (1 .9) 


610 (2.1) 


Indiana State, US 


376 (10.4) 


451 (5.0) 


502 (4.7) 


555 (5.9) 


627 (8.7) 


Ontario Province, Can. 


403 (9.6) 


479 (3.6) 


531 (3.8) 


577 (3.5) 


641 (4.8) 


Quebec Province, Can. 


429 (6.3) 


494 (3.5) 


540 (2.8) 


583 (2.9) 


646 (11.0) 



( ) Standard errors appear in parentheses. 
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Exhibit D.3: Percentiles of Achievement in Reasoning Cognitive Domain 



MATHEMATICS 

Grade 



Countries 


5th 

Percentile 


25th 

Percentile 


50th 

Percentile 


75th 

Percentile 


95th 

Percentile 


Armenia 


305 (4.4) 


387 (3.9) 


445 (2.8) 


503 (3.9) 


588 (4.7) 


Australia 


373 (5.5) 


455 (5.4) 


510 (4.9) 


561 (3.4) 


633 (5.5) 


Belgium (Flemish) 


421 (4.3) 


494 (3.3) 


542 (3.0) 


591 (3.5) 


659 (4.3) 


Chinese Taipei 


422 (7.8) 


513 (3.4) 


568 (2.9) 


618 (2.0) 


691 (5.9) 


Cyprus 


369 (4.5) 


460 (3.2) 


519 (2.6) 


574 (3.3) 


653 (4.4) 


England 


390 (8.1) 


479 (4.6) 


539 (3.3) 


598 (4.0) 


678 (4.9) 


Hong Kong, SAR 


427 (6.4) 


514 (4.4) 


568 (4.4) 


618 (3.0) 


689 (3.9) 


Hungary 


379 (5.6) 


467 (4.7) 


528 (2.9) 


584 (4.0) 


661 (4.3) 


Iran, Islamic Rep. of 


267 (9.9) 


346 (4.5) 


400 (4.2) 


455 (3.4) 


531 (3.9) 


Italy 


356 (4.7) 


441 (3.0) 


499 (3.4) 


556 (3.7) 


644 (6.0) 


Japan 


423 (7.2) 


508 (1.6) 


563 (2.9) 


618 (3.5) 


694 (4.4) 


Latvia 


393 (7.5) 


476 (2.7) 


533 (2.6) 


588 (3.6) 


664 (5.2) 


Lithuania 


381 (4.9) 


469 (3.5) 


530 (3.5) 


586 (4.1) 


662 (5.9) 


Moldova, Rep. of 


341 (9.7) 


436 (4.7) 


498 (6.1) 


556 (5.5) 


636 (5.9) 


Morocco 


235 (5.7) 


314 (5.3) 


368 (4.5) 


423 (6.5) 


502 (6.2) 


Netherlands 


422 (4.8) 


492 (3.2) 


536 (5.3) 


580 (4.2) 


643 (3.4) 


New Zealand 


360 (3.7) 


448 (2.4) 


507 (3.4) 


561 (3.3) 


637 (5.3) 


Norway 


322 (5.5) 


409 (2.4) 


471 (2.0) 


527 (5.1) 


608 (3.5) 


Philippines 


201 (6.9) 


287 (4.5) 


352 (6.5) 


425 (10.1) 


544 (18.4) 


Russian Federation 


386 (6.3) 


468 (5.1) 


526 (4.9) 


585 (5.0) 


665 (4.5) 


Scotland 


366 (7.2) 


448 (4.3) 


500 (3.7) 


551 (4.1) 


623 (7.1) 


Singapore 


403 (10.9) 


516 (7.7) 


579 (5.3) 


640 (8.3) 


722 (5.3) 


Slovenia 


339 (8.7) 


429 (3.7) 


488 (3.2) 


543 (4.7) 


619 (4.3) 


Tunisia 


179 (8.5) 


273 (3.8) 


339 (5.2) 


406 (4.8) 


504 (6.2) 


United States 


388 (3.4) 


466 (3.8) 


520 (2.5) 


574 (2.6) 


647 (2.9) 


Benchmarking Participants 


Indiana State, US 


416 (4.1) 


484 (3.1) 


529 (2.8) 


573 (4.8) 


639 (9.0) 


Ontario Province, Can. 


406 (4.4) 


476 (4.1) 


523 (3.8) 


571 (5.2) 


637 (8.0) 


Quebec Province, Can. 


389 (4.9) 


464 (4.2) 


514 (2.3) 


562 (2.6) 


629 (3.9) 



( ) Standard errors appear in parentheses. 
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Exhibit D.4: Standard Deviations of Achievement in Knowing Cognitive Domain 



MATHEMATICS L O ) 

Grade (^Oj 



C nnntripc 


Overall 


Girls 


Boys 


V.UUI III ICj 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Armenia 


480 (2.9) 


77 (1.3) 


486 (3.2) 


74 (1.3) 


474 (3.4) 


79 (1 .8) 


Australia 


497 (4.0) 


69 (2.6) 


491 (5.1) 


66 (2.3) 


502 (5.2) 


72 (3.8) 


Bahrain 


401 (2.3) 


83 (1.3) 


419 (2.8) 


77 (1.7) 


383 (3.2) 


85 (1 .9) 


Belgium (Flemish) 


537 (2.5) 


68 (2.2) 


534 (3.4) 


68 (3.0) 


541 (3.6) 


69 (2.5) 


Botswana 


372 (2.8) 


73 (1.7) 


374 (3.0) 


72 (1.7) 


370 (3.6) 


75 (2.1) 


Bulgaria 


486 (4.1) 


81 (2.4) 


487 (5.0) 


80 (3.0) 


484 (4.3) 


81 (2.9) 


Chile 


386 (3.2) 


77 (1.6) 


378 (3.5) 


74 (1.7) 


393 (4.1) 


79 (2.2) 


Chinese Taipei 


585 (4.5) 


98 (2.0) 


589 (5.0) 


92 (2.1) 


582 (5.0) 


103 (2.4) 


Cyprus 


466 (2.0) 


70 (0.8) 


474 (2.1) 


67 (1 .4) 


458 (2.6) 


71 (1.2) 


Egypt 


411 (3.4) 


92 (1.5) 


413 (4.1) 


89 (1 .5) 


409 (4.8) 


94 (2.3) 


England 


489 (4.0) 


62 (2.7) 


488 (4.5) 


61 (2.6) 


489 (5.2) 


63 (3.5) 


Estonia 


538 (2.7) 


63 (1 .4) 


538 (3.2) 


63 (1 .6) 


538 (3.0) 


63 (1 .6) 


Ghana 


232 (5.9) 


114 (2.7) 


220 (7.0) 


111 (3.1) 


243 (6.4) 


115 (3.0) 


Hong Kong, SAR 


589 (3.3) 


67 (3.0) 


591 (3.7) 


66 (3.0) 


587 (4.6) 


69 (3.5) 


Hungary 


536 (3.1) 


73 (1.7) 


537 (3.6) 


72 (2.0) 


536 (3.4) 


74 (2.1) 


Indonesia 


422 (4.3) 


84 (2.2) 


423 (4.5) 


84 (2.8) 


421 (4.6) 


84 (2.2) 


Iran, Islamic Rep. of 


405 (2.6) 


76 (1.2) 


412 (4.8) 


74 (1 .8) 


401 (4.2) 


77 (2.5) 


Israel 


501 (3.1) 


76 (1.7) 


499 (3.3) 


72 (1.7) 


503 (4.0) 


79 (2.3) 


Italy 


484 (3.2) 


74 (2.2) 


483 (3.2) 


72 (2.5) 


485 (3.8) 


76 (2.4) 


Japan 


564 (1 .9) 


71 (1.3) 


564 (3.8) 


68 (3.7) 


564 (3.4) 


74 (2.3) 


Jordan 


428 (4.7) 


94 (2.1) 


442 (5.5) 


91 (2.8) 


415 (6.2) 


94 (2.5) 


Korea, Rep. of 


592 (2.1) 


83 (1 .6) 


589 (2.9) 


81 (1.7) 


594 (2.4) 


85 (1 .8) 


Latvia 


518 (2.8) 


66 (1 .4) 


523 (2.9) 


63 (1 .6) 


514 (3.4) 


68 (1 .9) 


Lebanon 


447 (3.2) 


70 (1.5) 


444 (3.6) 


70 (1 .6) 


452 (3.9) 


71 (2.1) 


Lithuania 


511 (2.7) 


72 (1.4) 


514 (3.5) 


71 (1.8) 


507 (2.9) 


75 (1.7) 


Macedonia, Rep. of 


447 (3.8) 


87 (2.4) 


452 (4.1) 


85 (2.5) 


442 (4.2) 


88 (2.8) 


Malaysia 


506 (3.9) 


70 (2.1) 


511 (4.4) 


68 (2.5) 


501 (4.4) 


71 (2.4) 


Moldova, Rep. of 


466 (4.1) 


81 (1.9) 


472 (4.9) 


80 (2.2) 


460 (4.6) 


82 (2.2) 


Morocco 


386 (2.8) 


76 (1.5) 


383 (3.7) 


76 (2.1) 


392 (4.2) 


76 (2.6) 


Netherlands 


520 (3.1) 


58 (2.4) 


518 (3.5) 


58 (2.5) 


522 (3.6) 


59 (2.6) 


New Zealand 


485 (4.8) 


66 (3.0) 


484 (4.3) 


63 (2.8) 


486 (6.5) 


69 (3.6) 


Norway 


450 (2.1) 


57 (1.3) 


451 (2.5) 


55 (1.6) 


449 (3.0) 


58 (1 .6) 


Palestinian Nat 1 1 Auth. 


391 (3.7) 


100 (1.6) 


398 (4.8) 


98 (2.2) 


382 (5.6) 


100 (2.3) 


Philippines 


388 (5.2) 


88 (2.5) 


395 (5.2) 


87 (2.5) 


379 (6.1) 


90 (3.4) 


Romania 


485 (4.9) 


92 (1.9) 


489 (5.3) 


91 (2.0) 


481 (5.2) 


94 (2.4) 


Russian Federation 


519 (3.4) 


74 (1.3) 


525 (3.5) 


72 (1.5) 


514 (3.7) 


77 (1.6) 


Saudi Arabia 


315 (4.6) 


89 (2.1) 


309 (6.9) 


85 (4.0) 


319 (6.2) 


91 (2.5) 


Scotland 


481 (3.2) 


60 (1 .7) 


483 (3.9) 


58 (2.2) 


478 (3.4) 


61 (1.7) 


Serbia 


495 (2.7) 


80 (1 .6) 


502 (3.1) 


77 (1.9) 


489 (3.1) 


83 (1.7) 


Singapore 


591 (3.1) 


71 (2.1) 


596 (2.9) 


68 (2.5) 


586 (3.7) 


74 (2.2) 


Slovak Republic 


517 (3.3) 


75 (1.7) 


521 (3.4) 


72 (2.0) 


514 (3.9) 


78 (2.0) 


Slovenia 


499 (2.2) 


65 (1 .4) 


502 (2.6) 


62 (1 .6) 


495 (2.9) 


68 (1 .7) 


South Africa 


261 (5.4) 


106 (5.3) 


260 (6.1) 


104 (6.1) 


261 (6.3) 


109 (6.0) 


Sweden 


486 (2.1) 


55 (1.3) 


486 (2.5) 


54 (1 .6) 


486 (2.3) 


56 (1.5) 


Tunisia 


399 (3.0) 


69 (1 .4) 


388 (3.6) 


69 (1 .8) 


410 (3.0) 


66 (1 .5) 


United States 


510 (2.8) 


69 (1 .4) 


508 (3.0) 


67 (1.5) 


512 (3.0) 


71 (1.6) 


nchmarking Participants 














Basque Country, Spain 


495 (2.2) 


59 (1.2) 


498 (2.2) 


55 (1.1) 


492 (3.0) 


62 (2.0) 


Indiana State, US 


515 (4.6) 


61 (2.7) 


512 (4.7) 


60 (2.7) 


518 (5.2) 


63 (2.9) 


Ontario Province, Can. 


513 (2.6) 


57 (1.4) 


512 (2.9) 


57 (1.7) 


513 (3.1) 


57 (1.6) 


Quebec Province, Can. 


537 (2.7) 


53 (1.5) 


535 (3.3) 


52 (1.7) 


540 (3.1) 


54 (1 .8) 



( ) Standard errors appear in parentheses. 
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Exhibit D.4: Standard Deviations of Achievement in Knowing Cognitive Domain 



MATHEMATICS 

Grade 




C m intripc 


Overall 


Girls 


Boys 


V_ U LI 1 1 11 ICj 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Armenia 


447 (3.7) 


91 (2.2) 


455 (3.7) 


88 (2.1) 


439 (4.3) 


93 (2.7) 


Australia 


501 (3.8) 


81 (2.2) 


501 (4.1) 


79 (2.3) 


502 (4.6) 


83 (3.0) 


Belgium (Flemish) 


558 (2.1) 


66 (1.1) 


556 (2.6) 


65 (1.3) 


560 (2.9) 


68 (1.5) 


Chinese Taipei 


565 (2.2) 


72 (1.4) 


564 (2.4) 


68 (1.8) 


566 (2.6) 


76 (1.6) 


Cyprus 


500 (2.8) 


88 (1.3) 


496 (3.1) 


86 (1.8) 


504 (3.6) 


89 (1.5) 


England 


534 (4.5) 


93 (2.1) 


534 (4.4) 


90 (2.4) 


534 (5.3) 


96 (2.5) 


Hong Kong, SAR 


574 (3.3) 


69 (1.3) 


574 (3.9) 


65 (1.3) 


573 (3.6) 


72 (2.0) 


Hungary 


517 (3.3) 


80 (2.0) 


516 (4.1) 


78 (2.6) 


518 (3.6) 


81 (2.1) 


Iran, Islamic Rep. of 


404 (4.0) 


78 (2.0) 


411 (6.6) 


78 (3.3) 


399 (4.5) 


78 (1.9) 


Italy 


514 (3.9) 


89 (2.1) 


510 (4.3) 


88 (2.8) 


518 (4.1) 


90 (2.0) 


Japan 


564 (2.1) 


82 (1.1) 


565 (2.6) 


78 (1.4) 


564 (3.0) 


86 (1.4) 


Latvia 


517 (2.9) 


67 (1.5) 


518 (3.1) 


64 (1.5) 


515 (3.5) 


70 (2.3) 


Lithuania 


519 (2.7) 


75 (1.3) 


522 (3.3) 


74 (1.9) 


520 (3.6) 


77 (1.4) 


Moldova, Rep. of 


500 (5.2) 


97 (3.2) 


507 (5.9) 


96 (3.9) 


495 (5.3) 


97 (3.2) 


Morocco 


360 (4.4) 


87 (1.5) 


356 (5.6) 


88 (1.9) 


363 (4.3) 


86 (2.2) 


Netherlands 


530 (2.2) 


54 (1.4) 


527 (3.0) 


54 (1.7) 


533 (2.6) 


54 (1.8) 


New Zealand 


493 (2.2) 


84 (2.2) 


494 (2.9) 


83 (2.6) 


492 (2.5) 


85 (2.4) 


Norway 


448 (2.1) 


77 (1.2) 


447 (2.5) 


75 (1.6) 


449 (3.1) 


80 (1.6) 


Philippines 


385 (6.9) 


99 (5.3) 


389 (8.0) 


100 (6.1) 


380 (6.3) 


98 (4.6) 


Russian Federation 


513 (5.3) 


83 (2.1) 


513 (5.8) 


83 (2.5) 


514 (5.8) 


84 (2.4) 


Scotland 


484 (3.0) 


76 (1.8) 


478 (3.1) 


72 (2.2) 


489 (4.2) 


79 (1.9) 


Singapore 


626 (6.5) 


104 (3.2) 


632 (6.4) 


98 (2.9) 


620 (7.2) 


108 (3.7) 


Slovenia 


470 (2.6) 


70 (1.4) 


469 (3.1) 


67 (1.7) 


471 (3.3) 


72 (1.8) 


Tunisia 


338 (4.2) 


91 (2.2) 


337 (4.8) 


90 (2.4) 


338 (4.6) 


91 (2.5) 


United States 


528 (2.5) 


79 (1.3) 


525 (2.4) 


77 (1.4) 


532 (3.0) 


81 (1.6) 


Benchmarking Participants 


Indiana State, US 


544 (3.7) 


72 (1.9) 


541 (4.4) 


69 (2.2) 


546 (4.3) 


75 (2.3) 


Ontario Province, Can. 


514 (4.4) 


73 (2.8) 


508 (4.4) 


70 (2.7) 


519 (5.7) 


76 (3.6) 


Quebec Province, Can. 


504 (2.8) 


67 (1.5) 


501 (3.2) 


67 (1.9) 


508 (2.9) 


68 (1.6) 



( ) Standard errors appear in parentheses. 
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Exhibit D.5: Standard Deviations of Achievement in Applying Cognitive Domain 



MATHEMATICS L O ) 

Grade (^Oj 



C nnntripc 


Overall 


Girls 


Boys 


V.UUI III ICj 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Armenia 


478 (3.0) 


86 (1 .5) 


482 (3.5) 


83 (1 .8) 


473 (3.5) 


88 (2.0) 


Australia 


508 (4.8) 


86 (3.0) 


501 (6.1) 


82 (3.0) 


516 (6.0) 


88 (4.2) 


Bahrain 


398 (1 .6) 


76 (1.2) 


411 (2.3) 


71 (1.1) 


384 (2.3) 


79 (1.7) 


Belgium (Flemish) 


536 (2.7) 


72 (2.2) 


529 (3.3) 


71 (2.8) 


544 (3.7) 


73 (2.6) 


Botswana 


369 (2.7) 


71 (1.7) 


370 (3.0) 


70 (2.0) 


368 (2.9) 


71 (1.7) 


Bulgaria 


471 (4.7) 


87 (2.7) 


471 (6.0) 


87 (3.5) 


472 (4.9) 


87 (3.0) 


Chile 


391 (3.3) 


84 (1 .8) 


382 (3.6) 


81 (1.8) 


399 (4.2) 


86 (2.4) 


Chinese Taipei 


582 (4.6) 


101 (1.8) 


584 (5.1) 


96 (1 .9) 


580 (5.1) 


107 (2.4) 


Cyprus 


457 (1 .6) 


88 (1.3) 


465 (1 .9) 


85 (1.5) 


450 (2.5) 


90 (1 .8) 


Egypt 


404 (3.4) 


96 (1 .5) 


401 (4.3) 


93 (1.7) 


406 (4.9) 


98 (2.2) 


England 


503 (4.8) 


79 (3.1) 


503 (5.4) 


78 (2.9) 


504 (6.0) 


80 (4.2) 


Estonia 


528 (2.9) 


71 (1.4) 


531 (3.3) 


71 (1.7) 


526 (3.2) 


71 (1.8) 


Ghana 


293 (4.0) 


82 (2.0) 


286 (4.9) 


79 (2.2) 


299 (4.8) 


83 (2.3) 


Hong Kong, SAR 


584 (3.2) 


73 (3.1) 


584 (3.7) 


70 (3.0) 


584 (4.5) 


75 (3.8) 


Hungary 


523 (3.4) 


82 (2.0) 


517 (3.8) 


80 (2.2) 


529 (4.0) 


83 (2.4) 


Indonesia 


408 (4.9) 


93 (2.5) 


408 (5.0) 


92 (3.1) 


409 (5.3) 


93 (2.5) 


Iran, Islamic Rep. of 


416 (2.5) 


77 (1.4) 


420 (4.6) 


74 (2.0) 


413 (4.1) 


78 (2.7) 


Israel 


495 (3.6) 


91 (2.0) 


490 (3.7) 


87 (2.4) 


500 (4.6) 


94 (2.4) 


Italy 


484 (3.2) 


79 (1 .9) 


479 (3.0) 


76 (2.1) 


488 (4.0) 


82 (2.4) 


Japan 


564 (2.2) 


82 (1.6) 


563 (4.4) 


78 (4.4) 


565 (3.6) 


85 (2.2) 


Jordan 


422 (4.2) 


89 (2.0) 


436 (4.9) 


86 (2.7) 


409 (5.8) 


90 (2.2) 


Korea, Rep. of 


584 (2.2) 


84 (1 .4) 


581 (2.9) 


82 (1.6) 


587 (2.3) 


86 (1 .6) 


Latvia 


504 (3.4) 


76 (1 .9) 


505 (3.5) 


74 (2.4) 


504 (4.1) 


79 (2.5) 


Lebanon 


426 (3.3) 


71 (1.6) 


422 (3.7) 


70 (1 .6) 


432 (4.2) 


72 (2.4) 


Lithuania 


499 (2.8) 


84 (1 .5) 


499 (3.2) 


82 (2.1) 


497 (3.3) 


87 (1 .8) 


Macedonia, Rep. of 


428 (3.8) 


89 (2.5) 


431 (4.2) 


87 (2.8) 


426 (4.3) 


91 (2.8) 


Malaysia 


512 (4.4) 


78 (2.2) 


515 (5.1) 


76 (2.7) 


508 (4.8) 


79 (2.5) 


Moldova, Rep. of 


457 (3.9) 


80 (2.0) 


462 (4.0) 


77 (2.4) 


453 (4.5) 


82 (2.3) 


Morocco 


384 (2.9) 


66 (1 .4) 


377 (3.4) 


66 (1 .9) 


393 (3.3) 


65 (2.0) 


Netherlands 


543 (3.7) 


69 (2.7) 


538 (4.0) 


68 (2.8) 


548 (4.3) 


69 (3.1) 


New Zealand 


497 (5.3) 


80 (3.2) 


496 (4.7) 


77 (2.9) 


497 (7.2) 


83 (3.7) 


Norway 


468 (2.7) 


74 (1 .4) 


469 (2.8) 


72 (1.5) 


468 (3.4) 


76 (2.0) 


Palestinian Nat 1 1 Auth. 


388 (3.2) 


91 (1.6) 


389 (4.1) 


91 (2.0) 


388 (4.6) 


91 (2.2) 


Philippines 


378 (4.8) 


83 (2.8) 


383 (4.8) 


82 (2.8) 


373 (5.5) 


85 (3.6) 


Romania 


475 (5.0) 


91 (1.8) 


475 (5.4) 


90 (2.1) 


474 (5.3) 


93 (2.3) 


Russian Federation 


503 (3.7) 


80 (1 .6) 


503 (3.8) 


78 (1 .8) 


503 (4.1) 


82 (2.2) 


Saudi Arabia 


338 (3.6) 


75 (2.2) 


332 (6.1) 


72 (3.2) 


344 (4.5) 


76 (2.5) 


Scotland 


505 (3.9) 


76 (2.2) 


506 (4.8) 


74 (2.8) 


504 (3.8) 


79 (2.3) 


Serbia 


467 (2.9) 


97 (1.5) 


468 (3.5) 


94 (2.1) 


466 (3.1) 


100 (1.6) 


Singapore 


611 (3.6) 


80 (2.5) 


617 (3.6) 


76 (2.8) 


606 (4.1) 


83 (2.6) 


Slovak Republic 


502 (3.7) 


87 (1 .8) 


499 (4.0) 


84 (2.0) 


505 (4.3) 


90 (2.2) 


Slovenia 


491 (2.3) 


74 (1.1) 


491 (3.0) 


71 (1.6) 


491 (2.8) 


77 (1.5) 


South Africa 


269 (5.3) 


106 (5.1) 


267 (5.9) 


103 (5.8) 


271 (6.5) 


109 (5.9) 


Sweden 


505 (2.8) 


76 (1 .6) 


504 (3.2) 


75 (1.8) 


506 (2.8) 


77 (2.0) 


Tunisia 


419 (2.3) 


62 (1.3) 


407 (2.6) 


61 (1.7) 


433 (2.4) 


60 (1 .3) 


United States 


502 (3.4) 


83 (1 .6) 


497 (3.5) 


81 (1.8) 


506 (3.5) 


85 (1.7) 


nchmarking Participants 














Basque Country, Spain 


481 (2.3) 


66 (1 .5) 


483 (2.5) 


62 (1.5) 


480 (3.2) 


70 (2.2) 


Indiana State, US 


507 (5.9) 


76 (3.0) 


498 (5.7) 


73 (3.2) 


516 (6.7) 


78 (3.6) 


Ontario Province, Can. 


522 (3.0) 


70 (1.1) 


520 (3.3) 


68 (1 .5) 


525 (3.5) 


71 (1.5) 


Quebec Province, Can. 


545 (3.0) 


58 (1.7) 


539 (3.6) 


57 (1.8) 


549 (3.4) 


59 (2.0) 



( ) Standard errors appear in parentheses. 
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Exhibit D.5: Standard Deviations of Achievement in Applying Cognitive Domain 



MATHEMATICS 

Grade 




C nnntripc 


Overall 


Girls 


Boys 


V.UUI 1 11 ICO 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Armenia 


462 (3.2) 


82 (1.9) 


465 (3.2) 


80 (2.0) 


459 (3.7) 


84 (2.2) 


Australia 


490 (3.8) 


83 (2.0) 


487 (4.3) 


81 (2.2) 


492 (4.4) 


86 (2.5) 


Belgium (Flemish) 


546 (2.1) 


60 (1.1) 


544 (2.5) 


59 (1.3) 


548 (2.7) 


62 (1.4) 


Chinese Taipei 


561 (1.9) 


60 (0.9) 


561 (2.0) 


56 (1.0) 


562 (2.2) 


64 (1.2) 


Cyprus 


510 (2.8) 


86 (1.5) 


504 (3.1) 


83 (1.6) 


516 (2.9) 


88 (1.9) 


England 


526 (4.1) 


93 (2.0) 


524 (4.1) 


90 (2.3) 


528 (4.9) 


96 (2.5) 


Hong Kong, SAR 


577 (3.3) 


66 (1.2) 


576 (3.5) 


63 (1.4) 


577 (3.5) 


69 (1.7) 


Hungary 


530 (3.4) 


79 (2.2) 


530 (4.0) 


78 (2.6) 


531 (3.7) 


79 (2.5) 


Iran, Islamic Rep. of 


391 (3.8) 


78 (1.7) 


391 (6.1) 


77 (2.7) 


391 (4.8) 


80 (1.9) 


Italy 


494 (3.6) 


83 (1.9) 


489 (4.3) 


82 (2.6) 


498 (3.5) 


84 (1.7) 


Japan 


566 (2.1) 


81 (1.5) 


563 (2.6) 


77 (1.8) 


569 (2.3) 


85 (1.8) 


Latvia 


545 (3.3) 


80 (1.7) 


545 (3.4) 


77 (2.1) 


546 (4.0) 


83 (2.7) 


Lithuania 


542 (2.9) 


76 (1.5) 


541 (3.7) 


75 (2.3) 


545 (3.7) 


79 (1.7) 


Moldova, Rep. of 


507 (4.8) 


90 (3.0) 


511 (5.3) 


89 (3.7) 


502 (5.0) 


90 (3.0) 


Morocco 


349 (4.5) 


83 (1.8) 


345 (5.6) 


83 (2.2) 


352 (4.4) 


83 (1.9) 


Netherlands 


541 (2.6) 


60 (1.3) 


538 (2.8) 


61 (2.0) 


545 (3.2) 


58 (1.8) 


New Zealand 


486 (2.3) 


86 (1.9) 


485 (3.0) 


85 (2.5) 


486 (2.5) 


87 (1.8) 


Norway 


446 (2.2) 


76 (1.5) 


443 (2.8) 


73 (1.5) 


449 (3.0) 


78 (2.1) 


Philippines 


364 (7.5) 


101 (6.3) 


370 (8.8) 


101 (6.8) 


357 (6.7) 


100 (5.9) 


Russian Federation 


542 (4.7) 


84 (2.0) 


539 (5.0) 


83 (2.3) 


545 (5.1) 


85 (2.5) 


Scotland 


487 (3.5) 


80 (1.5) 


482 (3.6) 


76 (1.8) 


492 (4.6) 


84 (1.9) 


Singapore 


595 (5.9) 


90 (3.3) 


599 (5.8) 


86 (3.0) 


590 (6.6) 


94 (3.8) 


Slovenia 


477 (2.8) 


77 (1.6) 


474 (3.1) 


74 (1.9) 


481 (3.8) 


80 (2.3) 


Tunisia 


348 (4.6) 


102 (2.3) 


351 (5.1) 


102 (2.5) 


346 (4.7) 


102 (2.6) 


United States 


505 (2.6) 


77 (1.3) 


501 (2.8) 


75 (1.2) 


510 (2.9) 


80 (1.7) 


Benchmarking Participants 


Indiana State, US 


523 (3.3) 


72 (2.5) 


521 (3.4) 


68 (2.5) 


525 (4.2) 


76 (2.9) 


Ontario Province, Can. 


498 (4.5) 


74 (3.4) 


491 (3.8) 


70 (2.5) 


505 (5.8) 


76 (4.4) 


Quebec Province, Can. 


498 (2.7) 


66 (1.2) 


493 (3.2) 


65 (1.7) 


503 (2.9) 


67 (1.4) 



( ) Standard errors appear in parentheses. 
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MATHEMATICS L O ) 

Grade (^Oj 



C nnntripc 


Overall 


Girls 


Boys 


V.UUI III ICj 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Armenia 


468 (2.8) 


100 (2.0) 


473 (3.4) 


98 (1 .9) 


463 (4.3) 


103 (2.5) 


Australia 


515 (4.0) 


82 (2.4) 


515 (5.1) 


80 (2.5) 


516 (5.1) 


84 (3.4) 


Bahrain 


424 (2.2) 


76 (1.3) 


435 (2.5) 


71 (1.7) 


412 (3.2) 


79 (1 .6) 


Belgium (Flemish) 


533 (2.8) 


76 (2.2) 


531 (3.8) 


75 (2.8) 


536 (3.6) 


77 (2.6) 


Botswana 


353 (3.7) 


91 (1.5) 


356 (3.5) 


90 (1 .8) 


351 (4.5) 


92 (1.9) 


Bulgaria 


471 (3.9) 


88 (2.5) 


471 (5.2) 


86 (2.9) 


471 (4.4) 


89 (2.9) 


Chile 


409 (3.5) 


91 (1.6) 


406 (4.1) 


90 (2.0) 


412 (4.2) 


92 (1.9) 


Chinese Taipei 


576 (4.2) 


93 (1 .9) 


581 (4.3) 


88 (2.0) 


572 (4.8) 


97 (2.3) 


Cyprus 


455 (1.7) 


87 (1.5) 


465 (2.3) 


83 (2.3) 


446 (2.4) 


90 (1 .7) 


Egypt 


400 (3.6) 


94 (1 .5) 


402 (4.5) 


92 (1.7) 


399 (5.1) 


95 (2.1) 


England 


509 (4.7) 


82 (2.9) 


513 (4.8) 


80 (2.5) 


506 (5.9) 


85 (3.8) 


Estonia 


523 (3.0) 


75 (1.7) 


526 (3.4) 


75 (2.0) 


519 (3.4) 


74 (2.1) 


Ghana 


313 (4.0) 


97 (1.5) 


309 (4.6) 


97 (2.4) 


317 (5.0) 


97 (1 .8) 


Hong Kong, SAR 


569 (3.1) 


76 (2.6) 


571 (3.5) 


73 (2.5) 


567 (4.4) 


78 (3.4) 


Hungary 


529 (3.1) 


77 (1.7) 


530 (3.7) 


76 (2.1) 


528 (3.5) 


79 (2.1) 


Indonesia 


406 (4.3) 


92 (2.3) 


405 (4.4) 


91 (3.0) 


406 (4.8) 


94 (2.2) 


Iran, Islamic Rep. of 


417 (2.8) 


75 (1.3) 


423 (3.8) 


72 (1.7) 


413 (4.4) 


77 (2.1) 


Israel 


483 (3.3) 


92 (1.8) 


483 (3.4) 


89 (2.6) 


483 (4.6) 


96 (2.9) 


Italy 


489 (2.9) 


76 (1.7) 


486 (3.0) 


74 (1 .9) 


491 (3.4) 


78 (2.1) 


Japan 


576 (1 .8) 


76 (1.3) 


575 (3.7) 


72 (3.0) 


576 (3.0) 


80 (2.0) 


Jordan 


433 (3.7) 


83 (1 .8) 


442 (4.1) 


81 (2.3) 


425 (5.3) 


83 (2.2) 


Korea, Rep. of 


582 (1.7) 


82 (1.1) 


580 (2.4) 


79 (1 .6) 


584 (2.1) 


85 (1.3) 


Latvia 


500 (3.4) 


80 (1 .8) 


504 (3.6) 


76 (1.7) 


496 (4.4) 


83 (2.6) 


Lebanon 


410 (3.0) 


80 (2.1) 


407 (3.5) 


79 (2.1) 


413 (4.9) 


81 (2.8) 


Lithuania 


489 (2.6) 


81 (1.3) 


492 (3.0) 


79 (1 .6) 


484 (3.3) 


84 (2.1) 


Macedonia, Rep. of 


438 (3.7) 


93 (2.3) 


444 (4.1) 


90 (2.8) 


432 (4.7) 


95 (2.5) 


Malaysia 


503 (3.4) 


71 (1.5) 


505 (3.9) 


70 (1 .9) 


501 (3.9) 


73 (1.7) 


Moldova, Rep. of 


453 (4.0) 


86 (2.4) 


458 (4.2) 


85 (2.7) 


448 (4.6) 


86 (3.3) 


Morocco 


391 (3.2) 


80 (2.0) 


387 (3.9) 


80 (2.0) 


397 (4.0) 


80 (2.8) 


Netherlands 


541 (3.8) 


74 (2.5) 


540 (4.3) 


74 (2.5) 


542 (4.5) 


76 (2.8) 


New Zealand 


509 (5.2) 


81 (2.5) 


519 (5.4) 


76 (2.9) 


499 (6.7) 


85 (2.9) 


Norway 


479 (2.8) 


80 (1 .7) 


486 (3.1) 


78 (1 .9) 


472 (3.5) 


81 (2.1) 


Palestinian Nat 1 1 Auth. 


404 (2.7) 


87 (1 .6) 


410 (3.8) 


84 (1 .8) 


397 (4.2) 


89 (2.3) 


Philippines 


358 (5.8) 


110 (2.4) 


363 (5.9) 


109 (2.5) 


350 (6.4) 


112 (3.4) 


Romania 


458 (4.5) 


95 (2.0) 


460 (5.0) 


92 (2.8) 


456 (5.0) 


98 (2.3) 


Russian Federation 


496 (3.6) 


79 (1 .9) 


498 (4.0) 


77 (2.3) 


494 (3.8) 


81 (2.0) 


Saudi Arabia 


348 (4.3) 


87 (2.7) 


347 (5.7) 


82 (2.8) 


349 (6.1) 


90 (3.9) 


Scotland 


513 (3.4) 


81 (2.1) 


517 (4.3) 


79 (2.4) 


509 (3.4) 


82 (2.7) 


Serbia 


468 (2.6) 


86 (1 .0) 


472 (3.3) 


83 (1.5) 


464 (2.8) 


87 (1.3) 


Singapore 


583 (3.5) 


88 (2.2) 


589 (3.3) 


84 (2.7) 


579 (4.4) 


92 (2.5) 


Slovak Republic 


504 (3.2) 


83 (2.0) 


505 (3.3) 


80 (1 .8) 


503 (4.2) 


87 (2.6) 


Slovenia 


494 (2.5) 


72 (1.3) 


500 (3.1) 


68 (1 .9) 


488 (3.2) 


76 (1 .6) 


South Africa 


287 (5.0) 


118 (4.0) 


287 (5.6) 


116 (4.6) 


286 (5.7) 


121 (4.9) 


Sweden 


508 (3.3) 


84 (1 .9) 


511 (4.1) 


83 (2.2) 


505 (3.3) 


86 (2.6) 


Tunisia 


399 (2.7) 


71 (1.1) 


390 (3.3) 


71 (1.6) 


410 (3.3) 


69 (1 .6) 


United States 


505 (3.3) 


83 (1.5) 


505 (3.3) 


81 (1.7) 


506 (3.7) 


85 (1 .8) 


nchmarking Participants 














Basque Country, Spain 


494 (2.4) 


73 (1.5) 


500 (2.8) 


68 (1 .6) 


488 (3.8) 


77 (2.4) 


Indiana State, US 


503 (5.2) 


77 (3.0) 


502 (4.6) 


74 (2.6) 


503 (6.5) 


79 (3.7) 


Ontario Province, Can. 


527 (3.0) 


72 (1.5) 


527 (3.8) 


71 (2.0) 


528 (3.6) 


73 (1.8) 


Quebec Province, Can. 


539 (3.2) 


66 (1.7) 


537 (3.7) 


64 (2.2) 


540 (3.9) 


67 (1 .6) 



( ) Standard errors appear in parentheses. 
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MATHEMATICS 

Grade 




C nnntripc 


Overall 


Girls 


Boys 


V-UUI 1 11 ICj 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Mean 


Standard 

Deviation 


Armenia 


445 (3.1) 


86 (2.0) 


449 (3.4) 


84 (2.3) 


442 (3.4) 


87 (2.4) 


Australia 


507 (3.6) 


79 (1.8) 


507 (3.9) 


77 (2.1) 


507 (4.2) 


80 (2.3) 


Belgium (Flemish) 


541 (2.2) 


72 (1.5) 


541 (2.6) 


71 (2.0) 


541 (2.8) 


74 (1.7) 


Chinese Taipei 


563 (2.2) 


81 (1.3) 


565 (2.6) 


76 (1.6) 


562 (2.7) 


85 (1.7) 


Cyprus 


516 (2.4) 


86 (1.4) 


515 (2.7) 


83 (1.6) 


517 (3.0) 


88 (1.8) 


England 


537 (3.5) 


87 (1.5) 


539 (4.0) 


85 (2.0) 


536 (4.2) 


90 (2.1) 


Hong Kong, SAR 


564 (3.7) 


79 (1.4) 


565 (4.0) 


75 (1.7) 


563 (3.8) 


82 (2.0) 


Hungary 


524 (3.2) 


86 (2.0) 


525 (4.0) 


85 (2.4) 


524 (3.8) 


87 (2.3) 


Iran, Islamic Rep. of 


400 (3.4) 


80 (1.9) 


406 (6.0) 


79 (2.9) 


396 (4.3) 


81 (2.2) 


Italy 


499 (4.0) 


88 (2.0) 


496 (4.7) 


86 (2.7) 


502 (4.1) 


89 (2.1) 


Japan 


562 (1.7) 


83 (1.0) 


559 (2.1) 


79 (1.3) 


564 (2.6) 


86 (1.4) 


Latvia 


531 (3.2) 


83 (1.7) 


531 (3.3) 


80 (2.0) 


531 (4.1) 


85 (2.6) 


Lithuania 


526 (3.1) 


85 (1.8) 


527 (3.7) 


82 (2.5) 


529 (3.9) 


89 (2.0) 


Moldova, Rep. of 


494 (4.9) 


90 (3.1) 


501 (5.5) 


90 (3.9) 


488 (5.6) 


89 (3.3) 


Morocco 


368 (4.4) 


81 (2.1) 


366 (5.6) 


82 (3.1) 


370 (4.7) 


80 (2.1) 


Netherlands 


535 (2.9) 


67 (1.4) 


533 (3.4) 


66 (1.6) 


536 (3.2) 


68 (1.9) 


New Zealand 


503 (2.2) 


84 (1.7) 


502 (2.9) 


82 (2.2) 


504 (2.4) 


85 (1.9) 


Norway 


468 (2.1) 


87 (1.8) 


466 (2.5) 


85 (1.7) 


470 (2.8) 


88 (2.8) 


Philippines 


359 (7.4) 


104 (5.1) 


366 (8.8) 


106 (6.3) 


352 (6.6) 


101 (4.1) 


Russian Federation 


526 (4.8) 


85 (1.9) 


524 (5.2) 


83 (2.0) 


528 (4.9) 


86 (2.6) 


Scotland 


498 (3.1) 


78 (1.9) 


495 (3.5) 


75 (2.2) 


502 (4.0) 


81 (2.2) 


Singapore 


574 (6.1) 


96 (3.5) 


578 (6.2) 


92 (3.4) 


570 (6.8) 


99 (4.0) 


Slovenia 


485 (2.6) 


84 (1.9) 


486 (3.0) 


81 (2.6) 


485 (3.6) 


88 (2.4) 


Tunisia 


340 (4.2) 


98 (2.2) 


340 (5.8) 


98 (2.7) 


339 (4.7) 


98 (2.3) 


United States 


519 (2.5) 


78 (1.0) 


517 (2.6) 


76 (1.0) 


522 (2.9) 


80 (1.3) 


Benchmarking Participants 


Indiana State, US 


528 (3.4) 


67 (2.0) 


528 (3.9) 


64 (2.1) 


528 (4.1) 


70 (2.5) 


Ontario Province, Can. 


523 (3.6) 


70 (2.2) 


520 (3.4) 


67 (1.7) 


525 (4.8) 


73 (3.0) 


Quebec Province, Can. 


512 (2.6) 


73 (1.2) 


510 (3.0) 


72 (1.8) 


514 (3.2) 


73 (1.6) 



( ) Standard errors appear in parentheses. 
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